Effective Parallel Corpus Mining using Bilingual Sentence Embeddings
Mandy Guoa𝑎a, Qinlan Shenb𝑏b , Yinfei Yanga𝑎a, Heming Gea𝑎a, Daniel Cera𝑎a, Gustavo Hernandez Abregoa𝑎a, Keith Stevensa𝑎a, Noah Constanta𝑎a, Yun-Hsuan Sunga𝑎a, Brian Stropea𝑎a, Ray Kurzweila𝑎a \ANDa𝑎aGoogle AI Mountain View, CA, USA、b𝑏bCarnegie Mellon University Pittsburgh, PA, USA equal contribution Work done during an internship at Google AI.
Abstract
This paper presents an effective approach for parallel corpus mining using bilingual sentence embeddings. Our embedding models are trained to produce similar representations exclusively for bilingual sentence pairs that are translations of each other. This is achieved using a novel training method that introduces hard negatives consisting of sentences that are not translations but that have some degree of semantic similarity. The quality of the resulting embeddings are evaluated on parallel corpus reconstruction and by assessing machine translation systems trained on gold vs. mined sentence pairs. We find that the sentence embeddings can be used to reconstruct the United Nations Parallel Corpus Ziemski et al. (2016) at the sentence level with a precision of 48.9% for en-fr and 54.9% for en-es. When adapted to document level matching, we achieve a parallel document matching accuracy that is comparable to the significantly more computationally intensive approach of Uszkoreit et al. (2010). Using reconstructed parallel data, we are able to train NMT models that perform nearly as well as models trained on the original data (within 1-2 BLEU).
中文速览
想从海量未对齐的多语言文本中自动挖掘出高质量的平行句对，传统方法要么计算开销巨大、要么依赖大量人工特征工程。本文提出了一种基于双语句子嵌入（bilingual sentence embeddings）的轻量级挖掘方案，核心创新在于训练时引入"难负例（hard negatives）"——那些语义相近但并非真正互译的句子对，迫使模型学会区分真正的翻译关系与仅仅话题相似的句对。实验显示，在联合国平行语料库上，该方法以48.9%（英法）和54.9%（英西）的精度完成句级重建，文档级匹配精度接近计算量远超本文方法的经典系统，而基于挖掘数据训练的神经机器翻译模型与用原始人工对齐数据训练的模型相比，BLEU值差距仅在1–2分以内。这项工作表明，用简单高效的句子编码器配合精心设计的负例采样策略，就能以极低的计算成本获得接近工业级的平行数据挖掘效果，对低资源语言的机器翻译数据扩充具有重要的实用价值。
原文 arXiv:1807.11906;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1807.11906v2