Semi-Autoregressive Neural Machine Translation
Chunqi Wang Ji Zhang Haiqing Chen Alibaba Group {shiyuan.wcq, zj122146, Part of this work was done when the author was at Institute of Automation, Chinese Academy of Sciences.
Abstract
Existing approaches to neural machine translation are typically autoregressive models. While these models attain state-of-the-art translation quality, they are suffering from low parallelizability and thus slow at decoding long sequences. In this paper, we propose a novel model for fast sequence generation — the semi-autoregressive Transformer (SAT). The SAT keeps the autoregressive property in global but relieves in local and thus is able to produce multiple successive words in parallel at each time step. Experiments conducted on English-German and Chinese-English translation tasks show that the SAT achieves a good balance between translation quality and decoding speed. On WMT’14 English-German translation, the SAT achieves 5.58 $\times$ speedup while maintains 88% translation quality, significantly better than the previous non-autoregressive methods. When produces two words at each time step, the SAT is almost lossless (only 1% degeneration in BLEU score).
中文速览
自回归翻译模型（autoregressive model）每次只能生成一个词，解码速度慢，而完全并行的非自回归模型虽然快但翻译质量损失严重——本文提出的半自回归Transformer（Semi-Autoregressive Transformer, SAT）正是为了在两者之间找到更好的平衡点。SAT的核心思路是把目标序列按固定大小K分组，组间保持自回归依赖，组内的K个词则同时并行生成，为此还引入了"长距离预测"和"松弛因果掩码"两个关键改动。在WMT'14英德翻译任务上，SAT实现了5.58倍的解码加速，同时保留了88%的翻译质量，远优于此前的非自回归方法；当K=2时，BLEU分数几乎没有损失（仅下降约1%）。这项工作表明，适度放弃局部自回归性就能获得可观的速度提升，且该方法不局限于机器翻译，可推广到摘要生成、图像描述等任意序列生成任务。
原文 arXiv:1808.08583;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1808.08583v2