Training generative neural networks via Maximum Mean Discrepancy optimization
Gintare Karolina Dziugaite University of Cambridge、Daniel M. Roy University of Toronto、Zoubin Ghahramani University of Cambridge
Abstract
We consider training a deep neural network to generate samples from an unknown distribution given i.i.d. data. We frame learning as an optimization minimizing a two-sample test statistic—informally speaking, a good generator network produces samples that cause a two-sample test to fail to reject the null hypothesis. As our two-sample test statistic, we use an unbiased estimate of the maximum mean discrepancy, which is the centerpiece of the nonparametric kernel two-sample test proposed by Gretton et al. [2]. We compare to the adversarial nets framework introduced by Goodfellow et al. [1], in which learning is a two-player game between a generator network and an adversarial discriminator network, both trained to outwit the other. From this perspective, the MMD statistic plays the role of the discriminator. In addition to empirical comparisons, we prove bounds on the generalization error incurred by optimizing the empirical MMD.
中文速览
用神经网络生成逼真样本时，传统的生成对抗网络（GAN）需要同时训练生成器和判别器两个网络，调参繁琐、训练耗时长达数小时。本文提出用最大均值差异（Maximum Mean Discrepancy，MMD）这一有闭合解的核函数统计量直接替代判别器网络，将生成模型的训练转化为最小化一个可解析计算的两样本检验统计量的优化问题。由此得到的"MMD网络"只需对单一网络做梯度下降，省去了生成器与判别器之间反复博弈的复杂平衡，在MNIST和人脸数据集上均能快速收敛并生成质量可观的样本。此外，作者还从理论上证明了用经验MMD代替总体MMD时的泛化误差上界，为方法的可靠性提供了保障——这对于理解深度生成模型的学习理论具有重要意义。
原文 arXiv:1505.03906;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1505.03906v1