Self-Attention Generative Adversarial Networks
Han Zhang Ian Goodfellow Dimitris Metaxas Augustus Odena
Abstract
In this paper, we propose the Self-Attention Generative Adversarial Network (SAGAN) which allows attention-driven, long-range dependency modeling for image generation tasks. Traditional convolutional GANs generate high-resolution details as a function of only spatially local points in lower-resolution feature maps. In SAGAN, details can be generated using cues from all feature locations. Moreover, the discriminator can check that highly detailed features in distant portions of the image are consistent with each other. Furthermore, recent work has shown that generator conditioning affects GAN performance. Leveraging this insight, we apply spectral normalization to the GAN generator and find that this improves training dynamics. The proposed SAGAN performs better than prior work111Brock et al. (2018), which builds heavily on this work, has since improved those results substantially., boosting the best published Inception score from 36.8 to 52.52 and reducing Fréchet Inception distance from 27.62 to 18.65 on the challenging ImageNet dataset. Visualization of the attention layers shows that the generator leverages neighborhoods that correspond to object shapes rather than local regions
中文速览
卷积神经网络在生成图像时只能捕捉局部区域的依赖关系，导致GAN在合成结构复杂的图像（比如有清晰四肢的狗）时表现很差。研究者提出了自注意力生成对抗网络（Self-Attention GAN, SAGAN），在生成器和判别器中同时引入自注意力（self-attention）机制，让模型在生成每个像素细节时都能参考图像中任意远距离位置的信息，而不再局限于固定的局部感受野。此外，他们还将频谱归一化（spectral normalization）同时应用于生成器和判别器，并配合非对称学习率策略来稳定训练过程。在极具挑战性的ImageNet数据集上，SAGAN将当时最佳的Inception得分从36.8大幅提升至52.52，并将Fréchet Inception距离（FID）从27.62降低至18.65，可视化结果也证实模型学会了根据物体形状而非固定方块区域来分配注意力。这项工作证明了全局依赖建模对高质量图像生成的关键作用，并直接启发了后续BigGAN等里程碑式工作。
原文 arXiv:1805.08318;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1805.08318v2