EdiTTS: Score-based Editing for Controllable Text-to-Speech
Abstract
We present EdiTTS, an off-the-shelf speech editing methodology based on score-based generative modeling for text-to-speech synthesis. EdiTTS allows for targeted, granular editing of audio, both in terms of content and pitch, without the need for any additional training, task-specific optimization, or architectural modifications to the score-based model backbone. Specifically, we apply coarse yet deliberate perturbations in the Gaussian prior space to induce desired behavior from the diffusion model while applying masks and softening kernels to ensure that iterative edits are applied only to the target region. Through listening tests and speech-to-text back transcription, we show that EdiTTS outperforms existing baselines and produces robust samples that satisfy user-imposed requirements.111Code and audio samples at https://editts.github.io.
中文速览
语音合成系统生成的结果往往难以精细控制音调和内容，EdiTTS 针对这一痛点，提出了一种无需额外训练或修改模型结构的语音编辑框架，直接复用预训练的基于分数的生成模型（score-based generative model，即扩散模型）来完成编辑任务。具体做法是：先对梅尔频谱先验（mel-spectrogram prior）施加粗粒度扰动来近似用户期望的音调或内容改变，再配合二值掩码和梯度软化核，让扩散模型在反向去噪过程中只对目标区域进行修正，从而生成自然流畅的编辑结果。在音调调整和内容替换两项任务上，EdiTTS 在主观听感评分和语音转文字回译准确率方面均优于 FastPitch、WORLD 等基线方法，且支持对单个词级别的精细操控。这项工作的意义在于，它证明了扩散模型本身就具备强大的"理解并修复粗糙编辑"的能力，开箱即用、无需重新训练，为实际语音后期制作提供了一条简洁可行的新路径。
原文 arXiv:2110.02584;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2110.02584v3