FastSpeech 2: Fast and High-Quality End-to-End Text to Speech
Yi Ren1, Chenxu Hu, Xu Tan2, Tao Qin2, Sheng Zhao3, Zhou Zhao1, Tie-Yan Liu2 1Zhejiang University 2Microsoft Research Asia 3Microsoft Azure Speech Authors contribute equally to this work.Corresponding author
Abstract
Non-autoregressive text to speech (TTS) models such as FastSpeech (Ren et al., 2019) can synthesize speech significantly faster than previous autoregressive models with comparable quality. The training of FastSpeech model relies on an autoregressive teacher model for duration prediction (to provide more information as input) and knowledge distillation (to simplify the data distribution in output), which can ease the one-to-many mapping problem (i.e., multiple speech variations correspond to the same text) in TTS. However, FastSpeech has several disadvantages: 1) the teacher-student distillation pipeline is complicated and time-consuming, 2) the duration extracted from the teacher model is not accurate enough, and the target mel-spectrograms distilled from teacher model suffer from information loss due to data simplification, both of which limit the voice quality. In this paper, we propose FastSpeech 2, which addresses the issues in FastSpeech and better solves the one-to-many mapping problem in TTS by 1) directly training the model with ground-truth target instead of the simplified output from teacher, and 2) introducing more variation information of speech (e.g., pitch, energy and
中文速览
语音合成（TTS）长期面临"一对多映射"难题——同一段文字可以对应无数种语调、节奏和音量各异的发音，非自回归模型FastSpeech虽然合成速度快，但它依赖繁琐的"师生蒸馏"流程来绕开这一难题，导致训练耗时长、音频质量因信息损失而受限。本文提出FastSpeech 2，直接用真实录音作为训练目标，并额外从语音波形中提取时长、音高（pitch）和能量（energy）作为条件输入，从而同时简化流程、补全文本与语音之间的信息缺口；其中音高采用连续小波变换转为频谱域进行预测，以应对音高剧烈波动的挑战。实验结果显示，FastSpeech 2的训练速度比FastSpeech快3倍，合成音质超越FastSpeech，甚至媲美自回归模型；进一步扩展的FastSpeech 2s还能直接从文本并行生成语音波形，实现真正的端到端推理，推断速度更快。这项工作为高质量、低延迟、易控制的语音合成提供了更实用的技术路径。
原文 arXiv:2006.04558;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2006.04558v8