Complete Multilingual Neural Machine Translation
Markus Freitag and Orhan Firat Google Research
Abstract
Multilingual Neural Machine Translation (MNMT) models are commonly trained on a joint set of bilingual corpora which is acutely English-centric (i.e. English either as the source or target language). While direct data between two languages that are non-English is explicitly available at times, its use is not common. In this paper, we first take a step back and look at the commonly used bilingual corpora (WMT), and resurface the existence and importance of implicit structure that existed in it: multi-way alignment across examples (the same sentence in more than two languages). We set out to study the use of multi-way aligned examples to enrich the original English-centric parallel corpora. We reintroduce this direct parallel data from multi-way aligned corpora between all source and target languages. By doing so, the English-centric graph expands into a complete graph, every language pair being connected. We call MNMT with such connectivity pattern complete Multilingual Neural Machine Translation (cMNMT) and demonstrate its utility and efficacy with a series of experiments and analysis. In combination with a novel training data sampling strategy that is conditioned on the target lan
中文速览
多语言神经机器翻译（Multilingual NMT）模型通常只在以英语为中心的双语语料上训练，导致非英语语言对之间几乎没有直接训练数据，翻译质量很差。本文发现，常用的公开语料（如WMT）中天然藏着大量"多路对齐"数据——同一句话在多种语言里都有译文，只需把共享同一英语句子的不同语言译文配对，就能免费获得大量非英语语言对的直接平行数据，无需爬取任何新语料。基于这一思路，作者提出"完全多语言机器翻译"（complete MNMT，cMNMT），将原本以英语为中心的星形连接扩展成每个语言对都有训练数据的完全图，并配套设计了一种只按目标语言采样的层级式训练策略，从根本上缓解了英语被过度曝光的问题。实验表明，cMNMT在所有非英语语言对上均超越双语基线、标准多语言基线和桥接翻译，同时保持英语相关语言对的翻译质量，最终还成功扩展到涵盖111种语言、超过12000个语言对的大规模场景，为低资源多语言翻译提供了一条实用且可扩展的新路径。
原文 arXiv:2010.10239;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2010.10239v1