Nearest Neighbor Machine Translation
Urvashi Khandelwal†, Angela Fan‡, Dan Jurafsky†, Luke Zettlemoyer‡, Mike Lewis‡ †Stanford University ‡Facebook AI Research Work done while the first author was interning at Facebook AI Research.
Abstract
We introduce $k$ -nearest-neighbor machine translation ( $k$ NN-MT), which predicts tokens with a nearest neighbor classifier over a large datastore of cached examples, using representations from a neural translation model for similarity search. This approach requires no additional training and scales to give the decoder direct access to billions of examples at test time, resulting in a highly expressive model that consistently improves performance across many settings. Simply adding nearest neighbor search improves a state-of-the-art German-English translation model by 1.5 BLEU. $k$ NN-MT allows a single model to be adapted to diverse domains by using a domain-specific datastore, improving results by an average of 9.2 BLEU over zero-shot transfer, and achieving new state-of-the-art results—without training on these domains. A massively multilingual model can also be specialized for particular language pairs, with improvements of 3 BLEU for translating from English into German and Chinese. Qualitatively, $k$ NN-MT is easily interpretable; it combines source and target context to retrieve highly relevant examples.
中文速览
传统神经机器翻译模型一旦训练完成就固化了参数，很难灵活适配新领域或特定语言对。研究者提出了 k近邻机器翻译（k-nearest-neighbor machine translation，kNN-MT）：在不修改、不重新训练原有神经翻译模型的前提下，把大量平行语料的上下文表示缓存成一个"数据库"，推理时直接用模型的隐层表示检索最相似的历史例子，把检索得到的词分布与原模型的输出分布加权融合，从而生成最终译文。实验表明，这一方法在多个场景下都带来了显著提升：对最先进的德英翻译模型提升了1.5个BLEU分；在领域适配场景中，仅换用特定领域的数据库，平均比零样本迁移高出9.2个BLEU，甚至超过了在该领域专门训练过的模型；多语言模型针对特定语言对也能提升约3个BLEU。这项工作的价值在于，它提供了一条无需任何额外训练、成本极低却效果显著的方式来增强和定制翻译系统，且检索结果本身可直接解释，为非参数方法在条件生成任务中的应用开辟了新思路。
原文 arXiv:2010.00710;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2010.00710v2