Deep Voice 2: Multi-Speaker Neural Text-to-Speech
Sercan Ö. Arık∗、Gregory Diamos∗ \ANDAndrew Gibiansky∗、John Miller∗、Kainan Peng∗、Wei Ping∗、Jonathan Raiman∗、Yanqi Zhou、Baidu Silicon Valley Artificial Intelligence Lab 1195 Bordeaux Dr. Sunnyvale, CA 94089 Listed alphabetically.
Abstract
We introduce a technique for augmenting neural text-to-speech (TTS) with low-dimensional trainable speaker embeddings to generate different voices from a single model. As a starting point, we show improvements over the two state-of-the-art approaches for single-speaker neural TTS: Deep Voice 1 and Tacotron. We introduce Deep Voice 2, which is based on a similar pipeline with Deep Voice 1, but constructed with higher performance building blocks and demonstrates a significant audio quality improvement over Deep Voice 1. We improve Tacotron by introducing a post-processing neural vocoder, and demonstrate a significant audio quality improvement. We then demonstrate our technique for multi-speaker speech synthesis for both Deep Voice 2 and Tacotron on two multi-speaker TTS datasets. We show that a single neural TTS system can learn hundreds of unique voices from less than half an hour of data per speaker, while achieving high audio quality synthesis and preserving the speaker identities almost perfectly.
中文速览
让单个语音合成模型同时发出数百种不同人的声音，同时还能保持每个人独特的音色——这正是本文要解决的核心难题。研究者提出了一种低维可训练说话人嵌入（speaker embedding）技术：为每位说话人分配一个小小的向量，将其注入神经网络的多个关键位置（如循环层初始状态、特征门控等），使模型在几乎完全共享参数的前提下学会区分不同声音。在此基础上，他们还升级了两个单说话人基线系统——推出了在架构上全面改进的 Deep Voice 2，并为 Tacotron 引入了基于 WaveNet 的神经声码器（vocoder）来替换原有的 Griffin-Lim 方法，两者的音频质量均获得显著提升。实验结果表明，一个模型可以从每位说话人不足半小时的数据中学会数百种独特声音，平均意见分（MOS）和说话人识别准确率均表现优异，说话人身份几乎被完整保留。这项工作意义在于大幅降低了多说话人语音合成的数据门槛，为低资源场景下快速扩展声音库提供了切实可行的路径。
原文 arXiv:1705.08947;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1705.08947v2