Quality Estimation without Human-labeled Data
Yi-Lin Tuan1, Ahmed El-Kishky2, Adithya Renduchintala2, Vishrav Chaudhary2, Francisco Guzmán2 and Lucia Specia3 1University of California Santa Barbara, 2Facebook AI, 3Imperial College London
Abstract
Quality estimation aims to measure the quality of translated content without access to a reference translation. This is crucial for machine translation systems in real-world scenarios where high-quality translation is needed. While many approaches exist for quality estimation, they are based on supervised machine learning requiring costly human labelled data. As an alternative, we propose a technique that does not rely on examples from human-annotators and instead uses synthetic training data. We train off-the-shelf architectures for supervised quality estimation on our synthetic data and show that the resulting models achieve comparable performance to models trained on human-annotated data, both for sentence and word-level prediction.
中文速览
翻译质量估计（Quality Estimation, QE）旨在不借助参考译文的情况下自动判断机器翻译的好坏，但现有主流方法依赖大量人工标注数据，成本高昂。为此，研究者提出了一套合成训练数据的流程：先用神经机器翻译（NMT）模型对挖掘所得的维基百科双语语料进行翻译，再用掩码语言模型（Masked Language Model, MLM）对参考译文进行随机改写以注入错误，然后通过编辑距离对齐自动生成词级和句级的伪标签，从而完全绕开人工标注。实验表明，在WMT20质量估计基准上，用合成数据训练的现成QE模型（NMT合成数据与MLM改写数据的集成）在词级和句级预测上均能达到乃至超越人工标注数据训练模型的水平。这一成果意味着在低资源或人工标注难以获取的场景下，无需人工后期编辑也能构建出高质量的QE系统，极大地降低了部署门槛。
原文 arXiv:2102.04020;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2102.04020v1