Noise Estimation for Generative Diffusion Models
Robin San Roman*1, Eliya Nachmani* 2,3, Lior Wolf 2
Abstract
Generative diffusion models have emerged as leading models in speech and image generation. However, in order to perform well with a small number of denoising steps, a costly tuning of the set of noise parameters is needed. In this work, we present a simple and versatile learning scheme that can step-by-step adjust those noise parameters, for any given number of steps, while the previous work needs to retune for each number separately. Furthermore, without modifying the weights of the diffusion model, we are able to significantly improve the synthesis results, for a small number of steps. Our approach comes at a negligible computation cost.
中文速览
扩散模型（Diffusion Model）生成质量高但推理步数多、速度慢，现有方法若想用少量步骤也能生成高质量样本，必须针对每种步数单独做昂贵的噪声调度（noise schedule）参数搜索。本文提出一种轻量级方案：额外训练一个小型噪声估计网络，在推理过程中实时估计当前数据的实际噪声水平，并据此动态调整后续去噪步骤的噪声调度参数，而无需修改原有扩散模型的权重。这一设计让同一套方法可以自适应地服务于任意步数设定，不再需要逐一重新调参。实验在图像生成（FID指标）和语音合成（PESQ、STOI指标）两个领域均验证了该方法的有效性：在步数受限的情况下，生成质量显著优于依赖人工网格搜索的基线方法，而额外计算开销几乎可以忽略不计。
原文 arXiv:2104.02600;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2104.02600v2