Massively Multilingual ASR: 50 Languages, 1 Model, 1 Billion Parameters
Abstract
We study training a single acoustic model for multiple languages with the aim of improving automatic speech recognition (ASR) performance on low-resource languages, and overall simplifying deployment of ASR systems that support diverse languages. We perform an extensive benchmark on 51 languages, with varying amount of training data by language (from 100 hours to 1100 hours). We compare three variants of multilingual training from a single joint model without knowing the input language, to using this information, to multiple heads (one per language “cluster”). We show that multilingual training of ASR models on several languages can improve recognition performance, in particular, on low resource languages. We see 20.9%, 23% and 28.8% average WER relative reduction compared to monolingual baselines on joint model, joint model with language input and multi head model respectively. To our knowledge, this is the first work studying multi-lingual ASR at massive scale, with more than 50 languages and more than 16,000 hours of audio across them.
中文速览
多语言语音识别（ASR）系统在实际部署时面临两大难题：为每种语言单独训练和维护模型成本极高，而低资源语言因训练数据稀少导致识别效果很差。研究者在51种语言、逾16000小时音频上大规模测试了三种多语言联合训练方案——共享所有参数的联合模型、输入语言标识的联合模型，以及按语言族群分配多个解码器头的多头模型，并引入课程学习和语言采样平衡等技巧来解决数据量差异悬殊带来的训练难题。实验结果显示，与各语言独立训练的基线相比，三种方案分别带来了20.9%、23%和28.8%的平均词错率（WER）相对下降，低资源语言的提升尤为显著，多头模型甚至在高资源语言上也保持了竞争力。这项工作首次在如此大规模（50+语言）的条件下系统验证了多语言联合训练的有效性，为用单一模型覆盖更多语言、降低系统维护成本提供了有力支撑。
原文 arXiv:2007.03001;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2007.03001v2