Fully Character-Level Neural Machine Translation without Explicit Segmentation
Jason Lee ETH Zürich、Kyunghyun Cho New York University、Thomas Hofmann ETH Zürich The majority of this work was completed while the author was visiting New York University.
Abstract
Most existing machine translation systems operate at the level of words, relying on explicit segmentation to extract tokens. We introduce a neural machine translation (NMT) model that maps a source character sequence to a target character sequence without any segmentation. We employ a character-level convolutional network with max-pooling at the encoder to reduce the length of source representation, allowing the model to be trained at a speed comparable to subword-level models while capturing local regularities. Our character-to-character model outperforms a recently proposed baseline with a subword-level encoder on WMT’15 DE-EN and CS-EN, and gives comparable performance on FI-EN and RU-EN. We then demonstrate that it is possible to share a single character-level encoder across multiple languages by training a model on a many-to-one translation task. In this multilingual setting, the character-level encoder significantly outperforms the subword-level encoder on all the language pairs. We observe that on CS-EN, FI-EN and RU-EN, the quality of the multilingual character-level translation even surpasses the models specifically trained on that language pair alone, both in terms of BLE
中文速览
纯字符级神经机器翻译长期受制于序列过长、训练极慢的问题，这项研究提出了一套完全不依赖任何分词的字符到字符（char2char）翻译模型，源端和目标端都直接在字符序列上操作。关键创新在于编码器：先用多尺度卷积层加最大池化大幅压缩源端字符序列长度，再接公路网络（highway network）和双向GRU，从而在保留局部字符规律的同时把计算量控制在可接受范围内，训练速度与子词（subword）级模型相当。在WMT'15的德英、捷英翻译任务上，该模型超过了以BPE子词为编码器的基线系统，在芬英和俄英上也持平；更突出的是多语言实验——用一个共享字符编码器同时翻译四种语言到英语时，字符级模型在所有语言对上都显著优于子词级基线，捷英、芬英、俄英三对甚至超过了各自专门训练的单语言模型，BLEU分数和人工评估均如此。这项工作证明字符级表示天然适合跨语言参数共享，无需为每种语言单独维护词表，为构建高参数效率的多语言翻译系统提供了一条切实可行的路径。
原文 arXiv:1610.03017;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1610.03017v3