Joint Training for Pivot-based Neural Machine Translation
Yong Cheng#, Yang Liu†, Qian Yang#, Maosong Sun† and Wei Xu# #Institute for Interdisciplinary Information Sciences, Tsinghua University, Beijing, China †State Key Laboratory of Intelligent Technology and Systems Tsinghua National Laboratory for Information Science and Technology Department of Computer Science and Technology, Tsinghua University, Beijing, China {liuyang2011, weixu,
Abstract
While recent neural machine translation approaches have delivered state-of-the-art performance for resource-rich language pairs, they suffer from the data scarcity problem for resource-scarce language pairs. Although this problem can be alleviated by exploiting a pivot language to bridge the source and target languages, the source-to-pivot and pivot-to-target translation models are usually independently trained. In this work, we introduce a joint training algorithm for pivot-based neural machine translation. We propose three methods to connect the two models and enable them to interact with each other during training. Experiments on Europarl and WMT corpora show that joint training of source-to-pivot and pivot-to-target models leads to significant improvements over independent training across various languages.
中文速览
低资源语言对（如西班牙语→法语）因缺乏直接平行语料，通常借助中间语言（枢轴语言，pivot language）分两步翻译，但这两步模型各自独立训练，导致错误逐级传播、模型之间不匹配。针对这一问题，研究者提出了一种"联合训练（joint training）"框架，通过三种方式把源语言→枢轴语言和枢轴语言→目标语言两个模型连接起来：强制共享枢轴语言词向量（硬匹配）、惩罚词向量间的欧氏距离（软匹配），以及利用少量源-目标桥接语料直接优化端到端翻译概率。在Europarl和WMT两个语料库上的实验表明，联合训练相比独立训练在多种语言组合上均取得了显著的BLEU提升。这项工作的意义在于，它为低资源机器翻译提供了一个无需大量直接平行语料、却能让两段翻译模型相互"感知"并协同优化的实用方案。
原文 arXiv:1611.04928;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1611.04928v2