Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation
Melvin Johnson, Mike Schuster, Quoc V. Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat Fernanda Viégas, Martin Wattenberg, Greg Corrado, Macduff Hughes, Jeffrey Dean
Abstract
We propose a simple solution to use a single Neural Machine Translation (NMT) model to translate between multiple languages. Our solution requires no changes to the model architecture from a standard NMT system but instead introduces an artificial token at the beginning of the input sentence to specify the required target language. The rest of the model, which includes an encoder, decoder and attention module, remains unchanged and is shared across all languages. Using a shared wordpiece vocabulary, our approach enables Multilingual NMT using a single model without any increase in parameters, which is significantly simpler than previous proposals for Multilingual NMT. On the WMT’14 benchmarks, a single multilingual model achieves comparable performance for English $\rightarrow$ French and surpasses state-of-the-art results for English $\rightarrow$ German. Similarly, a single multilingual model surpasses state-of-the-art results for French $\rightarrow$ English and German $\rightarrow$ English on WMT’14 and WMT’15 benchmarks, respectively. On production corpora, multilingual models of up to twelve language pairs allow for better translation of many individual pairs. In addition to
中文速览
用一个神经机器翻译模型同时处理多种语言，以往要么大改模型结构、要么为每种语言单独训练一个模型，代价高昂且难以扩展。谷歌研究团队提出了一个极简方案：只需在输入句子开头加一个特殊标记（如"<2es>"表示"翻译成西班牙语"），完全不改动编码器、解码器和注意力机制，就能让同一个模型处理任意多个语言对。实验表明，这一方法在 WMT 英法、英德等标准测试集上达到甚至超越了当时最优的单语言对模型，在谷歌内部最多包含十二个语言对的生产数据上也显著提升了低资源语言的翻译质量。更令人惊喜的是，模型还能"零样本翻译"——即对训练时从未出现过的语言对（如葡萄牙语→西班牙语）给出合理译文，说明模型内部自发形成了跨语言的通用语义表示，这为迁移学习在机器翻译中的应用提供了早期有力证据。
原文 arXiv:1611.04558;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1611.04558v2