Fast Spectrogram Inversion using Multi-head Convolutional Neural Networks
Sercan Ö. Arık∗, Heewoo Jun∗, Gregory Diamos Baidu Silicon Valley Artificial Intelligence Lab 1195 Bordeaux Dr. Sunnyvale, CA 94089.∗Equal contributionManuscript received August, 2018.
Abstract
We propose the multi-head convolutional neural network (MCNN) for waveform synthesis from spectrograms. Nonlinear interpolation in MCNN is employed with transposed convolution layers in parallel heads. MCNN enables significantly better utilization of modern multi-core processors than commonly-used iterative algorithms like Griffin-Lim, and yields very fast (more than 300x real-time) runtime. For training of MCNN, we use a large-scale speech recognition dataset and losses defined on waveforms that are related to perceptual audio quality. We demonstrate that MCNN constitutes a very promising approach for high-quality speech synthesis, without any iterative algorithms or autoregression in computations.
中文速览
从语谱图（spectrogram）重建音频波形是语音合成流程中的关键一步，传统的 Griffin-Lim 等迭代算法速度慢、无法针对特定场景优化，研究者提出了一种名为多头卷积神经网络（Multi-head CNN，MCNN）的非自回归模型来解决这一问题。MCNN 由多个并行的转置卷积头组成，各头学习不同的上采样核并协同工作，同时引入了谱收敛、对数幅度、瞬时频率和加权相位四种与感知音质相关的损失函数进行训练。在 960 小时的 LibriSpeech 数据集上训练后，MCNN 的主观评分（MOS）超过了 Griffin-Lim 算法，说话人分类准确率也保持在高水平，同时在 GPU 上实现了超过 300 倍实时速度的波形合成，比 GL 快约 20–60 倍。这项工作证明无需自回归也能实现高质量语音合成，并为语音克隆、语音增强等端到端音频处理系统提供了一个快速、可训练、易集成的波形生成模块。
原文 arXiv:1808.06719;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1808.06719v2