Exploring Speech Enhancement with Generative Adversarial Networks for Robust Speech Recognition
Abstract
We investigate the effectiveness of generative adversarial networks (GANs) for speech enhancement, in the context of improving noise robustness of automatic speech recognition (ASR) systems. Prior work [1] demonstrates that GANs can effectively suppress additive noise in raw waveform speech signals, improving perceptual quality metrics; however this technique was not justified in the context of ASR. In this work, we conduct a detailed study to measure the effectiveness of GANs in enhancing speech contaminated by both additive and reverberant noise. Motivated by recent advances in image processing [2], we propose operating GANs on log-Mel filterbank spectra instead of waveforms, which requires less computation and is more robust to reverberant noise. While GAN enhancement improves the performance of a clean-trained ASR system on noisy speech, it falls short of the performance achieved by conventional multi-style training (MTR). By appending the GAN-enhanced features to the noisy inputs and retraining, we achieve a $7$ % WER improvement relative to the MTR system.
中文速览
生成对抗网络（GAN）能否用来提升语音识别（ASR）在噪声环境下的准确率，这正是这项研究试图回答的问题。研究者先测试了在原始波形上做增强的 SEGAN 方法，发现它对混响噪声效果很差，甚至拖累识别性能；为此他们提出了 FSEGAN，改为在对数梅尔滤波器组频谱（log-Mel filterbank spectra）上做生成对抗增强，计算量更小、对混响也更鲁棒，使纯净语音训练的 ASR 系统错词率（WER）相对降低了 54%。不过单独使用 GAN 增强仍不及业界通行的多风格训练（multi-style training, MTR）方案；真正的突破来自将 GAN 增强特征与原始噪声特征拼接后重新训练 MTR 模型，在 MTR 基础上再相对减少了 7% 的 WER。这一结果说明 GAN 增强与 MTR 并不互斥，两者结合可以挖掘出超越单一方法的噪声鲁棒性，为实际 ASR 系统设计提供了新的思路。
原文 arXiv:1711.05747;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1711.05747v2