Unsupervised Neural Machine Translation
Mikel Artetxe, Gorka Labaka、Eneko Agirre IXA NLP Group University of the Basque Country (UPV/EHU) \AndKyunghyun Cho New York University CIFAR Azrieli Global Scholar
Abstract
In spite of the recent success of neural machine translation (NMT) in standard benchmarks, the lack of large parallel corpora poses a major practical problem for many language pairs. There have been several proposals to alleviate this issue with, for instance, triangulation and semi-supervised learning techniques, but they still require a strong cross-lingual signal. In this work, we completely remove the need of parallel data and propose a novel method to train an NMT system in a completely unsupervised manner, relying on nothing but monolingual corpora. Our model builds upon the recent work on unsupervised embedding mappings, and consists of a slightly modified attentional encoder-decoder model that can be trained on monolingual corpora alone using a combination of denoising and backtranslation. Despite the simplicity of the approach, our system obtains 15.56 and 10.21 BLEU points in WMT 2014 French $\rightarrow$ English and German $\rightarrow$ English translation. The model can also profit from small parallel corpora, and attains 21.81 and 15.24 points when combined with 100,000 parallel sentences, respectively. Our implementation is released as an open source project111https:/
中文速览
仅凭单语语料库（无任何双语平行句对）训练神经机器翻译（NMT）系统，长期以来被认为几乎不可能实现。研究团队提出了一套完全无监督的训练框架：先用无监督方法将两种语言的词向量映射到同一空间，再以共享编码器加语言专属解码器构建翻译模型，通过"加噪重建"（denoising autoencoding）和"即时回译"（on-the-fly backtranslation）两种策略交替训练，让模型在没有任何对照翻译的情况下学会跨语言转换。在 WMT 2014 标准测试集上，该方法在法译英和德译英任务中分别达到 15.56 和 10.21 的 BLEU 分，若再补充 10 万句平行语料则进一步提升至 21.81 和 15.24。这项工作证明了"零平行数据"的神经机器翻译切实可行，为低资源语言对的机器翻译提供了全新的解决思路。
原文 arXiv:1710.11041;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1710.11041v2