Building Machine Translation Systems for the Next Thousand Languages
Ankur Bapna、Isaac Caswell††、Julia Kreutzer、Orhan Firat、Daan van Esch、Aditya Siddhant、Mengmeng Niu、Pallavi Baljekar、Xavier Garcia、Wolfgang Macherey、Theresa Breiner、Vera Axelrod、Jason Riesa、Yuan Cao、Mia Xu Chen、Klaus Macherey、Maxim Krikun、Pidong Wang、Alexander Gutkin、Apurva Shah、Yanping Huang、Zhifeng Chen、Yonghui Wu、Macduff Hughes Equal contributions. Correspondence to All authors affiliated with Google Research.
Abstract
In this paper we share findings from our effort to build practical machine translation (MT) systems capable of translating across over one thousand languages. We describe results in three research domains: (i) Building clean, web-mined datasets for 1500+ languages by leveraging semi-supervised pre-training for language identification and developing data-driven filtering techniques; (ii) Developing practical MT models for under-served languages by leveraging massively multilingual models trained with supervised parallel data for over $100$ high-resource languages and monolingual datasets for an additional $1000+$ languages; and (iii) Studying the limitations of evaluation metrics for these languages and conducting qualitative analysis of the outputs from our MT models, highlighting several frequent error modes of these types of models. Using this approach, we add 24 new languages to Google Translate, the product’s largest increase in language coverage to-date. We hope that our work provides useful insights to practitioners working towards building MT systems for currently understudied languages, and highlights research directions that can complement the weaknesses of massively multi
中文速览
大规模机器翻译长期只覆盖约100种语言，对非洲、南亚、美洲等地区的数千种语言几乎束手无策，谷歌这篇工作正是要打破这一瓶颈。研究团队分三步推进：首先用半监督语言识别（semi-supervised language identification）模型配合多种数据清洗过滤技术，从网络上为1500余种语言抓取出高质量单语语料；接着利用100余种高资源语言的有监督平行数据与上述单语数据联合训练超大规模多语言模型（massively multilingual model），并辅以回译（back-translation）和自训练（self-training）等数据增强手段，使模型在完全没有平行语料的"零资源"语言上也能产出可用的翻译；最后系统评估了现有指标（尤其是BLEU）在长尾语言上的局限性，并通过人工评测和错误分析揭示了模型易混淆近义词、低频词翻译退化、短文本处理较差等典型缺陷。最终，24种此前从未被支持的语言（包括博杰普尔语、奥罗莫语、盖丘亚语等）被加入了谷歌翻译，这是该产品历史上单次最大的语言扩展，也为业界提供了一套可复用的低资源多语言翻译建设路径。
原文 arXiv:2205.03983;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2205.03983v3