Segmental Recurrent Neural Networks for End-to-end Speech Recognition
Abstract
We study the segmental recurrent neural network for end-to-end acoustic modelling. This model connects the segmental conditional random field (CRF) with a recurrent neural network (RNN) used for feature extraction. Compared to most previous CRF-based acoustic models, it does not rely on an external system to provide features or segmentation boundaries. Instead, this model marginalises out all the possible segmentations, and features are extracted from the RNN trained together with the segmental CRF. Essentially, this model is self-contained and can be trained end-to-end. In this paper, we discuss practical training and decoding issues as well as the method to speed up the training in the context of speech recognition. We performed experiments on the TIMIT dataset. We achieved 17.3% phone error rate (PER) from the first-pass decoding — the best reported result using CRFs, despite the fact that we only used a zeroth-order CRF and without using any language model.
中文速览
分段循环神经网络（Segmental RNN）被用于端到端语音识别声学建模：它把分段条件随机场（Segmental CRF）与用于提取特征的循环神经网络（RNN）结合在一起，既不需要外部系统提供特征，也不需要预先给定分割边界，而是在训练时对所有可能的分割方案求边缘概率，CRF 与 RNN 联合优化。为解决计算量过大的问题，作者引入了最大分段长度限制和层级下采样 RNN，将训练速度提升约 10 倍。在 TIMIT 基准上，仅使用零阶 CRF、不借助任何语言模型，首次解码即达到 17.3% 的音素错误率，是目前基于 CRF 方法的最优结果。这项工作证明了分段 RNN 作为完全自包含的端到端声学模型的可行性，为摆脱 HMM 框架、实现更简洁的语音识别流水线提供了一条有竞争力的路径。
原文 arXiv:1603.00223;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1603.00223v2