EMOVIE: A Mandarin Emotion Speech Dataset with a Simple Emotional Text-to-Speech Model
Abstract
Recently, there has been an increasing interest in neural speech synthesis. While the deep neural network achieves the state-of-the-art result in text-to-speech (TTS) tasks, how to generate a more emotional and more expressive speech is becoming a new challenge to researchers due to the scarcity of high-quality emotion speech dataset and the lack of advanced emotional TTS model. In this paper, we first briefly introduce and publicly release a Mandarin emotion speech dataset including 9,724 samples with audio files and its emotion human-labeled annotation. After that, we propose a simple but efficient architecture for emotional speech synthesis called EMSpeech. Unlike those models which need additional reference audio as input, our model could predict emotion labels just from the input text and generate more expressive speech conditioned on the emotion embedding. In the experiment phase, we first validate the effectiveness of our dataset by an emotion classification task. Then we train our model on the proposed dataset and conduct a series of subjective evaluations. Finally, by showing a comparable performance in the emotional speech synthesis task, we successfully demonstrate the a
中文速览
如何让语音合成系统自动读出"情绪"，一直是个难题——现有数据集要么质量差、要么没有情感标注，现有模型又必须额外输入一段带情感的参考音频才能工作。为此，作者从七部普通话电影中整理出一份共9724条样本、时长约4.18小时的情感语音数据集EMOVIE，并用五级情感极性（−1到+1）进行人工标注，这是首个面向情感语音合成的公开普通话电影语音数据集。在此基础上，他们提出了EMSpeech模型，以FastSpeech 2为骨架，加入一个"情感预测器"——由ResNet风格的分类器从输入文本直接预测情感标签，再由Bi-LSTM情感控制器将标签转化为序列级情感嵌入，从而在不依赖任何参考音频的情况下控制语音的时长和音调，生成带有情感色彩的语音。实验表明，该数据集在情感分类任务上有效，EMSpeech在主观评测中取得与基线模型相当甚至更好的情感表现力，同时还支持手动指定情感类别来定向生成语音。这项工作的价值在于同时填补了中文情感语音数据和无参考音频情感合成模型两个空白，为更自然的人机对话和影视配音等应用打开了新的可能。
原文 arXiv:2106.09317;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2106.09317v1