f𝑓f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization
Sebastian Nowozin Machine Intelligence and Perception Group Microsoft Research Cambridge, UK、Botond Cseke Machine Intelligence and Perception Group Microsoft Research Cambridge, UK、Ryota Tomioka Machine Intelligence and Perception Group Microsoft Research Cambridge, UK
Abstract
Generative neural samplers are probabilistic models that implement sampling using feedforward neural networks: they take a random input vector and produce a sample from a probability distribution defined by the network weights. These models are expressive and allow efficient computation of samples and derivatives, but cannot be used for computing likelihoods or for marginalization. The generative-adversarial training method allows to train such models through the use of an auxiliary discriminative neural network. We show that the generative-adversarial approach is a special case of an existing more general variational divergence estimation approach. We show that any $f$ -divergence can be used for training generative neural samplers. We discuss the benefits of various choices of divergence functions on training complexity and the quality of the obtained generative models.
中文速览
生成对抗网络（GAN）的训练本质上是在最小化真实数据分布与生成模型之间的Jensen-Shannon散度，但这一框架能否推广到其他统计距离度量一直缺乏系统研究。本文将Nguyen等人提出的变分f-散度估计框架（variational f-divergence estimation）与GAN训练目标统一起来，证明GAN不过是这一更普遍框架的特例，并推导出适用于任意f-散度（f-divergence）的生成模型训练目标，涵盖KL散度、Pearson卡方散度等多种选择。在此基础上，作者提出了一种更简洁的单步梯度优化算法，并给出了其在鞍点邻域内几何收敛的理论保证。实验结果表明，当生成模型类与真实分布存在不匹配时，散度函数的选择对最终学到的模型有显著影响，为实践者根据任务需求灵活选取训练目标提供了理论依据与实用指导。
原文 arXiv:1606.00709;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1606.00709v1