Maximum-Likelihood Augmented Discrete Generative Adversarial Networks
Tong Che Yanran Li Ruixiang Zhang R Devon Hjelm Wenjie Li Yangqiu Song Yoshua Bengio
Abstract
Despite the successes in capturing continuous distributions, the application of generative adversarial networks (GANs) to discrete settings, like natural language tasks, is rather restricted. The fundamental reason is the difficulty of back-propagation through discrete random variables combined with the inherent instability of the GAN training objective. To address these problems, we propose Maximum-Likelihood Augmented Discrete Generative Adversarial Networks. Instead of directly optimizing the GAN objective, we derive a novel and low-variance objective using the discriminator’s output that follows corresponds to the log-likelihood. Compared with the original, the new objective is proved to be consistent in theory and beneficial in practice. The experimental results on various discrete datasets demonstrate the effectiveness of the proposed approach.
中文速览
用GAN训练离散序列（如自然语言）一直很棘手，因为离散采样无法直接反向传播，而直接用判别器输出作为强化学习奖励又极不稳定、方差极大。MaliGAN（最大似然增强离散生成对抗网络）的核心思路是：不再把判别器的输出直接当奖励，而是利用判别器构造一个近似真实数据分布的目标分布，再通过重要性采样把生成器的训练转化为对这个目标分布的最大似然优化，同时结合小批量归一化和基线等方差缩减技巧让梯度更稳定。理论上可以证明，当判别器训练充分时该目标等价于最小化生成分布与真实分布之间的KL散度，即便判别器未达最优也能保证生成器朝正确方向更新。在多个离散数据集上的实验表明，MaliGAN比直接用强化学习信号的方法更稳定、生成质量更好，为将GAN推广到自然语言生成等离散场景提供了一条切实可行的路径。
原文 arXiv:1702.07983;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1702.07983v1