Edinburgh Neural Machine Translation Systems for WMT 16
Rico Sennrich Barry Haddow Alexandra Birch School of Informatics, University of Edinburgh
Abstract
We participated in the WMT 2016 shared news translation task by building neural translation systems for four language pairs, each trained in both directions: English $\leftrightarrow$ Czech, English $\leftrightarrow$ German, English $\leftrightarrow$ Romanian and English $\leftrightarrow$ Russian. Our systems are based on an attentional encoder-decoder, using BPE subword segmentation for open-vocabulary translation with a fixed vocabulary. We experimented with using automatic back-translations of the monolingual News corpus as additional training data, pervasive dropout, and target-bidirectional models. All reported methods give substantial improvements, and we see improvements of 4.3–11.2 Bleu over our baseline systems. In the human evaluation, our systems were the (tied) best constrained system for 7 out of 8 translation directions in which we participated.111We have released the implementation that we used for the experiments as an open source toolkit: https://github.com/rsennrich/nematus222We have released scripts, sample configs, synthetic training data and trained models: https://github.com/rsennrich/wmt16-scripts
中文速览
机器翻译系统在处理低资源语言对或词表外词汇时往往表现不佳，爱丁堡大学团队在WMT 2016新闻翻译评测中针对这一挑战构建了基于注意力机制编码器-解码器（attentional encoder-decoder）的神经机器翻译系统，覆盖英语与捷克语、德语、罗马尼亚语、俄语四个语言对的双向翻译。他们综合运用了三项关键技术：用字节对编码（BPE，byte-pair encoding）将单词切分为子词单元以解决开放词表问题、将目标语言单语数据自动回译（back-translation）为伪双语数据来扩充训练集、以及对全部网络层（包括循环层）施加随机丢弃（pervasive dropout）来缓解过拟合，此外还训练了从右到左生成目标端的模型用于重排序。最终系统相比基线提升了4.3至11.2个BLEU分，并在参赛的8个翻译方向中有7个被评为人工评测最佳受限系统，充分证明这几项技术对神经机器翻译的实用价值，相关代码和数据也已开源供后续研究使用。
原文 arXiv:1606.02891;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1606.02891v2