Syllable-Based Sequence-to-Sequence Speech Recognition with the Transformer in Mandarin Chinese
Abstract
Sequence-to-sequence attention-based models have recently shown very promising results on automatic speech recognition (ASR) tasks, which integrate an acoustic, pronunciation and language model into a single neural network. In these models, the Transformer, a new sequence-to-sequence attention-based model relying entirely on self-attention without using RNNs or convolutions, achieves a new single-model state-of-the-art BLEU on neural machine translation (NMT) tasks. Since the outstanding performance of the Transformer, we extend it to speech and concentrate on it as the basic architecture of sequence-to-sequence attention-based model on Mandarin Chinese ASR tasks. Furthermore, we investigate a comparison between syllable based model and context-independent phoneme (CI-phoneme) based model with the Transformer in Mandarin Chinese. Additionally, a greedy cascading decoder with the Transformer is proposed for mapping CI-phoneme sequences and syllable sequences into word sequences. Experiments on HKUST datasets demonstrate that syllable based model with the Transformer performs better than CI-phoneme based counterpart, and achieves a character error rate (CER) of $28.77\%$ , which is c
中文速览
用Transformer这一完全依赖自注意力机制、不含RNN或卷积的序列到序列模型来做普通话自动语音识别（ASR），研究者同时比较了以音节（syllable，即带声调的拼音）和与上下文无关音素（CI-phoneme）作为建模单元时各自的表现，并提出了一种"贪心级联解码器"，通过两个串联的Transformer分别完成"语音→音节/音素序列"和"音节/音素序列→词序列"的转换。在150小时的HKUST普通话电话语音数据集上，音节建模方案明显优于音素方案，最终字错率（CER）达到28.77%，与当时最优的联合CTC-注意力编解码网络的28.0%相当。这项工作证明了Transformer强大的序列建模能力可以直接迁移到普通话ASR，同时揭示了音节作为建模单元在中文语音识别中的天然优势，为后续端到端中文语音识别研究提供了重要参考。
原文 arXiv:1804.10752;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1804.10752v2