Improved Denoising Diffusion Probabilistic Models
Alex Nichol Prafulla Dhariwal
Abstract
Denoising diffusion probabilistic models (DDPM) are a class of generative models which have recently been shown to produce excellent samples. We show that with a few simple modifications, DDPMs can also achieve competitive log-likelihoods while maintaining high sample quality. Additionally, we find that learning variances of the reverse diffusion process allows sampling with an order of magnitude fewer forward passes with a negligible difference in sample quality, which is important for the practical deployment of these models. We additionally use precision and recall to compare how well DDPMs and GANs cover the target distribution. Finally, we show that the sample quality and likelihood of these models scale smoothly with model capacity and training compute, making them easily scalable. We release our code at https://github.com/openai/improved-diffusion.
中文速览
扩散概率模型（Denoising Diffusion Probabilistic Model，DDPM）此前能生成高质量图像，却在对数似然（log-likelihood）这一衡量分布覆盖能力的关键指标上表现欠佳，也无法在少量采样步骤下高效生成图像。研究者通过三项改进解决了这些问题：让模型学习反向扩散过程的方差（而非固定不变）、将余弦噪声调度替换原有的线性调度、以及引入重要性采样来降低训练梯度噪声。这些改动使DDPM在ImageNet等高多样性数据集上的对数似然达到了与自回归模型相当的水平，同时把生成所需的前向推理次数从数百次压缩到约50次而几乎不损失图像质量；精确率/召回率（precision/recall）分析还表明，DDPM比GAN覆盖了更多目标分布的模式。这项工作证明扩散模型既能保持顶级样本质量，又具备良好的分布覆盖性和可扩展性，为其实际部署奠定了重要基础。
原文 arXiv:2102.09672;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2102.09672v1