State-of-the-art Speech Recognition With Sequence-to-Sequence Models
Abstract
Attention-based encoder-decoder architectures such as Listen, Attend, and Spell (LAS), subsume the acoustic, pronunciation and language model components of a traditional automatic speech recognition (ASR) system into a single neural network. In previous work, we have shown that such architectures are comparable to state-of-the-art ASR systems on dictation tasks, but it was not clear if such architectures would be practical for more challenging tasks such as voice search. In this work, we explore a variety of structural and optimization improvements to our LAS model which significantly improve performance. On the structural side, we show that word piece models can be used instead of graphemes. We also introduce a multi-head attention architecture, which offers improvements over the commonly-used single-head attention. On the optimization side, we explore synchronous training, scheduled sampling, label smoothing, and minimum word error rate optimization, which are all shown to improve accuracy. We present results with a unidirectional LSTM encoder for streaming recognition. On a $12,500$ hour voice search task, we find that the proposed changes improve the WER from $9.2$ % to $5.6$ %
中文速览
传统语音识别系统需要把声学模型、发音词典、语言模型分开构建，而"听、关注、拼写"（Listen, Attend and Spell，LAS）这类端到端神经网络试图用一个模型统一完成这一切，但此前在语音搜索这种更难的任务上仍落后于传统系统。研究者从模型结构和训练优化两个方向对LAS进行了系统性改进：结构上引入词片段（word piece model）作为输出单元、用多头注意力（multi-head attention）替换单头注意力；优化上采用最小词错率训练（MWER）、计划采样（scheduled sampling）、标签平滑（label smoothing）以及同步分布式训练，并在解码后用外部语言模型对N-best候选进行二次重排序。在一个1.25万小时的英语语音搜索任务上，改进后的LAS系统词错率从9.2%降至5.6%，而同期最强传统系统仅能达到6.7%，在听写任务上也同样超越传统系统。这一结果首次证明端到端序列到序列模型可以在大规模连续语音识别任务上全面超越传统混合系统，为纯神经网络语音识别的实用化提供了重要依据。
原文 arXiv:1712.01769;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1712.01769v6