Sequence-to-Sequence Speech Recognition with Time-Depth Separable Convolutions
Abstract
We propose a fully convolutional sequence-to-sequence encoder architecture with a simple and efficient decoder. Our model improves WER on LibriSpeech while being an order of magnitude more efficient than a strong RNN baseline. Key to our approach is a time-depth separable convolution block which dramatically reduces the number of parameters in the model while keeping the receptive field large. We also give a stable and efficient beam search inference procedure which allows us to effectively integrate a language model. Coupled with a convolutional language model, our time-depth separable convolution architecture improves by more than 22% relative WER over the best previously reported sequence-to-sequence results on the noisy LibriSpeech test set.
中文速览
语音识别领域的序列到序列（sequence-to-sequence）模型通常依赖多层双向LSTM编码器，参数量大、计算效率低，难以满足大规模训练的需求。本文提出了一种以"时深可分离卷积"（Time-Depth Separable Convolution，TDS）为核心的全卷积编码器，通过将时间维度的聚合与通道维度的混合部分解耦，在保持大感受野的同时大幅压缩参数量，并配合简化的解码器（去掉了位置注意力、输入反馈等串行依赖操作）和一套稳定高效的束搜索（beam search）推理流程。在960小时LibriSpeech基准测试上，TDS模型参数量仅为RNN基线的约四分之一（37M vs. 136M），却在嘈杂测试集上将词错率（WER）相对降低了22%以上，达到端到端系统的最优水平。这项工作表明，精心设计的卷积结构完全可以在效率和精度上同时超越传统RNN方案，对需要大规模半监督训练的实际语音系统具有重要意义。
原文 arXiv:1904.02619;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1904.02619v1