Revisiting Over-Smoothness in Text to Speech
Yi Ren Zhejiang University、Xu Tan Microsoft Research、Tao Qin Microsoft Research \ANDZhou Zhao Zhejiang University、Tie-Yan Liu Microsoft Research Corresponding author
Abstract
Non-autoregressive text to speech (NAR-TTS) models have attracted much attention from both academia and industry due to their fast generation speed. One limitation of NAR-TTS models is that they ignore the correlation in time and frequency domains while generating speech mel-spectrograms, and thus cause blurry and over-smoothed results. In this work, we revisit this over-smoothing problem from a novel perspective: the degree of over-smoothness is determined by the gap between the complexity of data distributions and the capability of modeling methods. Both simplifying data distributions and improving modeling methods can alleviate the problem. Accordingly, we first study methods reducing the complexity of data distributions. Then we conduct a comprehensive study on NAR-TTS models that use some advanced modeling methods. Based on these studies, we find that 1) methods that provide additional condition inputs reduce the complexity of data distributions to model, thus alleviating the over-smoothing problem and achieving better voice quality. 2) Among advanced modeling methods, Laplacian mixture loss performs well at modeling multimodal distributions and enjoys its simplicity, while GA
中文速览
非自回归文字转语音（Non-Autoregressive TTS，NAR-TTS）模型因生成速度快而备受关注，但其合成的语音频谱往往模糊、过于平滑，根本原因在于模型忽略了频谱在时间和频率维度上的相关性。本文从一个新视角重新审视这一"过平滑"问题：过平滑的严重程度取决于数据分布复杂性与建模方法能力之间的差距，因此既可以通过简化数据分布（如引入音高、时长、能量等额外条件信息），也可以通过增强建模方法（如使用拉普拉斯混合损失、GAN、Glow等）来缓解这一问题。研究发现，提供更多语音变量信息能有效降低分布复杂度，而GAN和Glow在主客观评测中均能达到最佳音质，将两类方法结合使用可进一步提升效果，且上述结论在多说话人场景中同样成立。这项系统性研究为社区设计更优秀的TTS模型提供了清晰的方向和实验依据。
原文 arXiv:2202.13066;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2202.13066v1