Improving End-to-End Speech Recognition with Policy Learning
Abstract
Connectionist temporal classification (CTC) is widely used for maximum likelihood learning in end-to-end speech recognition models. However, there is usually a disparity between the negative maximum likelihood and the performance metric used in speech recognition, e.g., word error rate (WER). This results in a mismatch between the objective function and metric during training. We show that the above problem can be mitigated by jointly training with maximum likelihood and policy gradient. In particular, with policy learning we are able to directly optimize on the (otherwise non-differentiable) performance metric. We show that joint training improves relative performance by 4% to 13% for our end-to-end model as compared to the same model learned through maximum likelihood. The model achieves 5.53% WER on Wall Street Journal dataset, and 5.42% and 14.70% on Librispeech test-clean and test-other set, respectively.
中文速览
端到端语音识别（end-to-end speech recognition）模型通常用最大似然方法训练，但训练目标和实际评价指标词错率（WER, word error rate）之间存在明显的鸿沟——模型把所有错误转录一视同仁，而WER却能区分"错多少"。为了弥合这一差距，作者将策略梯度（policy gradient）方法中的自批判序列训练（SCST, self-critical sequence training）与最大似然目标联合训练，让模型能直接优化原本不可微的WER指标，同时借助最大似然稳定早期训练、防止收敛困难。在模型结构上，他们采用深度可分离卷积加双向GRU的端到端架构，并配合数据增强与丰富的正则化手段。实验结果显示，联合训练在WSJ数据集上带来约14%的相对性能提升，在LibriSpeech上也有约4%的提升，最终在WSJ eval92上取得5.53% WER，在LibriSpeech test-clean和test-other上分别达到5.42%和14.70%，与当时最强方法不相上下。这项工作表明，用强化学习思路直接对评价指标"对症下药"是一条切实可行且计算代价不高的优化路径，对语音识别乃至其他序列生成任务均有参考价值。
原文 arXiv:1712.07101;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1712.07101v1