Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism
Orhan Firat Middle East Technical University、Kyunghyun Cho New York University、Yoshua Bengio University of Montreal CIFAR Senior Fellow
Abstract
We propose multi-way, multilingual neural machine translation. The proposed approach enables a single neural translation model to translate between multiple languages, with a number of parameters that grows only linearly with the number of languages. This is made possible by having a single attention mechanism that is shared across all language pairs. We train the proposed multi-way, multilingual model on ten language pairs from WMT’15 simultaneously and observe clear performance improvements over models trained on only one language pair. In particular, we observe that the proposed model significantly improves the translation quality of low-resource language pairs.
中文速览
用一个神经网络同时搞定多语言、多方向的机器翻译，听起来很美，但关键拦路虎在于：注意力机制（attention mechanism）过去被认为是语言对专属的，没法共享。本文提出了一种多路多语言神经机器翻译框架，核心创新是让所有语言对共用同一套注意力机制，同时为每种源语言配备独立编码器、每种目标语言配备独立解码器，参数量只随语言数量线性增长而非平方级膨胀。研究者在WMT'15的十个语言对上同时训练这一单一模型，结果表明共享注意力机制完全可行，尤其对低资源语言对的翻译质量有显著提升，说明多语言联合训练能让资源稀缺的语言对从其他语言对中"借力"。这项工作不仅证明了注意力机制可以跨语言通用，更为构建覆盖大量语言的统一翻译系统奠定了基础。
原文 arXiv:1601.01073;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1601.01073v1