Diffusion-GAN: Training GANs with Diffusion
Zhendong Wang1,2, Huangjie Zheng1,2, Pengcheng He2, Weizhu Chen2, Mingyuan Zhou1 1The University of Texas at Austin, 2Microsoft Azure AI {zhendong.wang,
Abstract
Generative adversarial networks (GANs) are challenging to train stably, and a promising remedy of injecting instance noise into the discriminator input has not been very effective in practice. In this paper, we propose Diffusion-GAN, a novel GAN framework that leverages a forward diffusion chain to generate Gaussian-mixture distributed instance noise. Diffusion-GAN consists of three components, including an adaptive diffusion process, a diffusion timestep-dependent discriminator, and a generator. Both the observed and generated data are diffused by the same adaptive diffusion process. At each diffusion timestep, there is a different noise-to-data ratio and the timestep-dependent discriminator learns to distinguish the diffused real data from the diffused generated data. The generator learns from the discriminator’s feedback by backpropagating through the forward diffusion chain, whose length is adaptively adjusted to balance the noise and data levels. We theoretically show that the discriminator’s timestep-dependent strategy gives consistent and helpful guidance to the generator, enabling it to match the true data distribution. We demonstrate the advantages of Diffusion-GAN over st
中文速览
GAN训练长期面临不稳定、模式崩溃等顽疾，向判别器输入加噪声（实例噪声）本是一种理论上可行的解法，但在高维图像上始终难以奏效。Diffusion-GAN提出用前向扩散链（forward diffusion chain）为真实图像和生成图像同时叠加结构化的高斯混合噪声，并设计了一个与扩散时间步绑定的判别器，让它在不同噪声强度下同时学会分辨真假；生成器则通过可微的扩散过程反向传播梯度来持续获得有效更新信号，扩散链的长度还会根据判别器的过拟合程度自适应调整。理论分析证明，这种时间步相关的判别策略能引导生成器最终匹配真实数据分布，而非陷入梯度消失的困境。在多个数据集上的实验表明，Diffusion-GAN显著提升了StyleGAN2等强基线的训练稳定性和数据效率，在FID和Recall等指标上达到当时最优水平，从而真正让"加噪声稳定GAN"这一旧思路在高分辨率图像生成上落地成功。
原文 arXiv:2206.02262;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2206.02262v4