Training Generative Adversarial Networks with Limited Data
Tero Karras NVIDIA、Miika Aittala NVIDIA、Janne Hellsten NVIDIA、Samuli Laine NVIDIA、Jaakko Lehtinen NVIDIA and Aalto University、Timo Aila NVIDIA
Abstract
Training generative adversarial networks (GAN) using too little data typically leads to discriminator overfitting, causing training to diverge. We propose an adaptive discriminator augmentation mechanism that significantly stabilizes training in limited data regimes. The approach does not require changes to loss functions or network architectures, and is applicable both when training from scratch and when fine-tuning an existing GAN on another dataset. We demonstrate, on several datasets, that good results are now possible using only a few thousand training images, often matching StyleGAN2 results with an order of magnitude fewer images. We expect this to open up new application domains for GANs. We also find that the widely used CIFAR-10 is, in fact, a limited data benchmark, and improve the record FID from 5.59 to 2.42.
中文速览
训练生成对抗网络（GAN）时，数据量不足会导致判别器过拟合，进而使训练崩溃——这篇论文正是要解决这一痛点。作者提出了一种「自适应判别器增强」（adaptive discriminator augmentation，ADA）机制：在训练过程中对所有输入判别器的图像施加一套随机变换（包括翻转、旋转、颜色调整等18种增强操作），并通过实时监测判别器的过拟合程度来动态调节增强强度，既无需修改损失函数或网络结构，也无需人工反复调参。关键创新在于理论上证明了只要每种变换以一定概率跳过，增强效果就不会「泄漏」到生成图像中，从而保证生成结果仍然干净自然。实验表明，仅用几千张图片便能达到此前需要数万张才能实现的StyleGAN2水准，CIFAR-10上的FID记录也从5.59大幅刷新至2.42，这意味着高质量GAN训练所需的数据门槛被显著降低，对医学影像等数据稀缺领域具有重要的现实意义。
原文 arXiv:2006.06676;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2006.06676v2