MTNT: A Testbed for Machine Translation of Noisy Text
Paul Michel Graham Neubig Language Technologies Institute Carnegie Mellon University
Abstract
Noisy or non-standard input text can cause disastrous mistranslations in most modern Machine Translation (MT) systems, and there has been growing research interest in creating noise-robust MT systems. However, as of yet there are no publicly available parallel corpora of with naturally occurring noisy inputs and translations, and thus previous work has resorted to evaluating on synthetically created datasets. In this paper, we propose a benchmark dataset for Machine Translation of Noisy Text (MTNT), consisting of noisy comments on Reddit111www.reddit.com and professionally sourced translations. We commissioned translations of English comments into French and Japanese, as well as French and Japanese comments into English, on the order of 7k-37k sentences per language pair. We qualitatively and quantitatively examine the types of noise included in this dataset, then demonstrate that existing MT models fail badly on a number of noise-related phenomena, even after performing adaptation on a small training set of in-domain data. This indicates that this dataset can provide an attractive testbed for methods tailored to handling noisy text in MT.222The data is publicly available at http:/
中文速览
机器翻译系统在处理社交媒体上那些充满拼写错误、缩写、表情符号和网络俚语的"脏文本"时往往会严重失准，但此前研究者既缺乏真实的噪声平行语料，也没有统一的评测基准。为此，作者从Reddit上收集了英语、法语、日语的真实嘈杂评论，委托专业译者完成双向翻译，构建了名为MTNT（Machine Translation of Noisy Text）的基准数据集，覆盖英法、法英、英日、日英四个语言对，每对约7k至37k句训练数据和约1000句测试数据。实验表明，现有主流神经机器翻译（Neural Machine Translation, NMT）模型在这类噪声文本上表现很差，即便用少量领域内数据做自适应微调，仍无法有效应对噪声带来的各类错误。这一数据集的公开发布，为研究如何让机器翻译真正用于真实网络环境提供了一个可靠的公共测试平台。
原文 arXiv:1809.00388;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1809.00388v1