Parallel WaveNet: Fast High-Fidelity Speech Synthesis
Aaron van den Oord, Yazhe Li, Igor Babuschkin, Karen Simonyan, Oriol Vinyals, Koray Kavukcuoglu \ANDGeorge van den Driessche, Edward Lockhart, Luis C. Cobo, Florian Stimberg, Norman Casagrande, Dominik Grewe, Seb Noury, Sander Dieleman, Erich Elsen, Nal Kalchbrenner, Heiga Zen, Alex Graves, Helen King, Tom Walters, Dan Belov, Demis Hassabis
Abstract
The recently-developed WaveNet architecture [27] is the current state of the art in realistic speech synthesis, consistently rated as more natural sounding for many different languages than any previous system. However, because WaveNet relies on sequential generation of one audio sample at a time, it is poorly suited to today’s massively parallel computers, and therefore hard to deploy in a real-time production setting. This paper introduces Probability Density Distillation, a new method for training a parallel feed-forward network from a trained WaveNet with no significant difference in quality. The resulting system is capable of generating high-fidelity speech samples at more than 20 times faster than real-time, and is deployed online by Google Assistant, including serving multiple English and Japanese voices.
中文速览
自动语音合成领域最强的WaveNet模型虽然听起来极为自然，却因为必须一个采样点一个采样点地顺序生成音频，在实际部署时慢得无法实用。为解决这一问题，研究者提出了"概率密度蒸馏"（Probability Density Distillation）方法：先用已训练好的WaveNet作为"教师"，再训练一个基于逆自回归流（Inverse Autoregressive Flow，IAF）的"学生"网络，让学生生成的样本在教师看来尽可能合理，同时保留足够的多样性，从而把慢速的自回归教师的知识迁移到可并行推理的学生模型中。最终，该并行WaveNet的生成速度比原始WaveNet快逾1000倍，在实时速度20倍以上即可产出高保真语音，且人类听感评测显示音质无明显下降，目前已被部署到Google Assistant的多个英语和日语语音服务中。这项工作首次真正打通了"训练快但生成慢"的自回归模型与"生成快但训练难"的流模型之间的壁垒，为高质量神经语音合成的大规模落地提供了切实可行的路径。
原文 arXiv:1711.10433;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1711.10433v1