Deep Voice: Real-time Neural Text-to-Speech
Sercan Ö. Arık† Mike Chrzanowski† Adam Coates† Gregory Diamos† Andrew Gibiansky† Yongguo Kang† Xian Li† John Miller† Andrew Ng† Jonathan Raiman† Shubho Sengupta† Mohammad Shoeybi† Baidu Silicon Valley Artificial Intelligence Lab, 1195 Bordeaux Dr. Sunnyvale, CA 94089
Abstract
We present Deep Voice, a production-quality text-to-speech system constructed entirely from deep neural networks. Deep Voice lays the groundwork for truly end-to-end neural speech synthesis. The system comprises five major building blocks: a segmentation model for locating phoneme boundaries, a grapheme-to-phoneme conversion model, a phoneme duration prediction model, a fundamental frequency prediction model, and an audio synthesis model. For the segmentation model, we propose a novel way of performing phoneme boundary detection with deep neural networks using connectionist temporal classification (CTC) loss. For the audio synthesis model, we implement a variant of WaveNet that requires fewer parameters and trains faster than the original. By using a neural network for each component, our system is simpler and more flexible than traditional text-to-speech systems, where each component requires laborious feature engineering and extensive domain expertise. Finally, we show that inference with our system can be performed faster than real time and describe optimized WaveNet inference kernels on both CPU and GPU that achieve up to 400x speedups over existing implementations.
中文速览
用纯深度神经网络造一套能在生产环境中实际使用的文字转语音（Text-to-Speech, TTS）系统，一直面临两大难题：传统系统各个模块都依赖繁琐的人工特征工程，而基于WaveNet的神经网络方案虽然音质接近人声，却慢得无法实时运行。Deep Voice把整条TTS流水线的五个环节——字素转音素、音素边界分割、音素时长预测、基频预测、音频合成——全部替换成神经网络，只保留音素、时长和基频这三类极简特征，其中分割模块创新性地引入CTC损失来检测音素边界，音频合成则采用改进版WaveNet（结合QRNN编码器替代转置卷积上采样），参数更少、训练更快。实验表明，系统能生成接近生产级别的语音质量，并通过对CPU和GPU推理内核的深度优化，实现了比此前WaveNet推理实现快达400倍的速度，真正做到了比实时更快的语音合成。这项工作的意义在于，它首次证明一套完全由神经网络构建、无需现有TTS系统辅助、只用音频和文本就能从头训练的系统，可以同时满足音质和实时性两个生产级要求，大幅降低了为新声音或新领域定制TTS系统的门槛。
原文 arXiv:1702.07825;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1702.07825v2