Quantitatively Evaluating GANs With Divergences Proposed for Training
Daniel Jiwoong Im1,2, He Ma3,4, Graham Taylor3,4,、Kristin Branson1 1Janelia Research Campus, HHMI, 2AIFounded Inc. 3University of Guelph, 4Vector Institute {imd, {hma02,
Abstract
Generative adversarial networks (GANs) have been extremely effective in approximating complex distributions of high-dimensional, input data samples, and substantial progress has been made in understanding and improving GAN performance in terms of both theory and application. However, we currently lack quantitative methods for model assessment. Because of this, while many GAN variants are being proposed, we have relatively little understanding of their relative abilities. In this paper, we evaluate the performance of various types of GANs using divergence and distance functions typically used only for training. We observe consistency across the various proposed metrics and, interestingly, the test-time metrics do not favour networks that use the same training-time criterion. We also compare the proposed metrics to human perceptual scores.
中文速览
如何客观衡量生成对抗网络（GAN）生成图像的质量，一直缺乏可靠的定量方法。研究者提出用训练GAN时常用的散度和距离函数——包括原始GAN准则、最小二乘GAN准则、Wasserstein距离和最大均值差异——作为独立的测试时评估指标，在测试集上重新训练一个"评判网络"来估计真实分布与生成分布之间的差距。实验在MNIST、CIFAR10等多个数据集上对DCGAN、LS-DCGAN、W-DCGAN三类框架进行了系统比较，发现四种指标的排名高度一致，且与人类感知判断的吻合度优于常用的Inception Score和FID；更有趣的是，某个GAN框架在测试时用与其训练目标不同的指标来衡量，结果排名依然稳定，说明"用什么指标训的网络在该指标下最好"这一直觉并不成立。这项工作为GAN模型选择、超参数调优提供了一套无需借助外部分类网络、更贴近生成质量本质的评估框架。
原文 arXiv:1803.01045;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1803.01045v2