Fast Decoding in Sequence Models Using Discrete Latent Variables
Łukasz Kaiser Aurko Roy Ashish Vaswani Niki Parmar Samy Bengio Jakob Uszkoreit Noam Shazeer
Abstract
Autoregressive sequence models based on deep neural networks, such as RNNs, Wavenet and the Transformer attain state-of-the-art results on many tasks. However, they are difficult to parallelize and are thus slow at processing long sequences. RNNs lack parallelism both during training and decoding, while architectures like WaveNet and Transformer are much more parallelizable during training, yet still operate sequentially during decoding. We present a method to extend sequence models using discrete latent variables that makes decoding much more parallelizable. We first auto-encode the target sequence into a shorter sequence of discrete latent variables, which at inference time is generated autoregressively, and finally decode the output sequence from this shorter latent sequence in parallel. To this end, we introduce a novel method for constructing a sequence of discrete latent variables and compare it with previously introduced methods. Finally, we evaluate our model end-to-end on the task of neural machine translation, where it is an order of magnitude faster at decoding than comparable autoregressive models. While lower in BLEU than purely autoregressive models, our model achieve
中文速览
传统的自回归序列模型（如RNN、Transformer）在生成文本时必须一个词一个词地顺序输出，导致解码速度极慢。研究者提出了"隐变量Transformer"（Latent Transformer）方法：先把目标序列压缩编码成一段更短的离散隐变量序列（长度缩短为原来的约1/8），再用自回归模型快速生成这段短序列，最后从短序列并行地还原出完整输出。为了更好地训练这种离散表示，他们还提出了一种新的向量量化方法——分解向量量化（Decomposed Vector Quantization，DVQ），有效解决了大码本下的"索引坍塌"问题，并与Gumbel-Softmax、VQ-VAE等现有方法做了系统比较。在机器翻译基准测试上，该模型的解码速度比同类自回归模型快约一个数量级，BLEU分数虽略低于纯自回归模型，但超越了此前所有非自回归翻译方案，且该框架无需针对翻译任务专门调参，可推广到任意序列生成问题。
原文 arXiv:1803.03382;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1803.03382v6