Modeling Coverage for Neural Machine Translation
Zhaopeng Tu† Zhengdong Lu† Yang Liu‡ Xiaohua Liu† Hang Li† †Noah’s Ark Lab, Huawei Technologies, Hong Kong ‡Department of Computer Science and Technology, Tsinghua University, Beijing
Abstract
Attention mechanism has enhanced state-of-the-art Neural Machine Translation (NMT) by jointly learning to align and translate. It tends to ignore past alignment information, however, which often leads to over-translation and under-translation. To address this problem, we propose coverage-based NMT in this paper. We maintain a coverage vector to keep track of the attention history. The coverage vector is fed to the attention model to help adjust future attention, which lets NMT system to consider more about untranslated source words. Experiments show that the proposed approach significantly improves both translation quality and alignment quality over standard attention-based NMT.111Our code is publicly available at https://github.com/tuzhaopeng/NMT-Coverage.
中文速览
神经机器翻译（NMT）中的注意力机制虽然能动态对齐源语言和目标语言，却会"忘记"之前已经翻译过哪些词，导致同一个源语言词被重复翻译（过翻译）或某些词根本没被翻译到（漏翻译）。为此，本文提出带覆盖机制的神经机器翻译（NMT-Coverage），核心思路是为每个源语言词维护一个覆盖向量（coverage vector），在解码每一步后累积更新注意力历史，并将该向量输入注意力模型，让模型在分配注意力时自动"记住"哪些词已被充分翻译、哪些还需关注。作者设计了两类覆盖模型：一类借鉴SMT中的词育率（fertility）概念，具有明确的语言学解释；另一类则用循环神经网络（RNN）以纯数据驱动的方式编码覆盖信息。实验证明，该方法在翻译质量和对齐质量上均显著优于标准注意力NMT，为解决NMT长期存在的过翻译与漏翻译问题提供了一个简洁而有效的框架。
原文 arXiv:1601.04811;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1601.04811v6