Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks
Jost Tobias Springenberg University of Freiburg 79110 Freiburg, Germany
Abstract
In this paper we present a method for learning a discriminative classifier from unlabeled or partially labeled data. Our approach is based on an objective function that trades-off mutual information between observed examples and their predicted categorical class distribution, against robustness of the classifier to an adversarial generative model. The resulting algorithm can either be interpreted as a natural generalization of the generative adversarial networks (GAN) framework or as an extension of the regularized information maximization (RIM) framework to robust classification against an optimal adversary. We empirically evaluate our method – which we dub categorical generative adversarial networks (or CatGAN) – on synthetic data as well as on challenging image classification tasks, demonstrating the robustness of the learned classifiers. We further qualitatively assess the fidelity of samples generated by the adversarial generator that is learned alongside the discriminative classifier, and identify links between the CatGAN objective and discriminative clustering algorithms (such as RIM).
中文速览
判别式分类器通常需要大量标注数据才能训练好，但现实中标注往往稀缺，如何从无标签或少标签数据中学出好的多类分类器是个长期难题。这篇论文提出了"类别生成对抗网络"（Categorical GAN，CatGAN），把经典的生成对抗网络（GAN）框架从二分类扩展到多类无监督/半监督分类：让判别器尽量对真实样本给出确定且均匀分布于各类的预测，同时对生成器造出的假样本保持不确定；生成器则反过来试图生成能骗过判别器、让其作出高置信度类别判断的样本，这种对抗机制天然充当了防止判别器过拟合虚假相关性的正则化手段。在合成数据、MNIST 和 CIFAR-10 上的实验表明，CatGAN 学到的分类器鲁棒性强、聚类质量高，同时对抗生成器也能产生视觉上可信的样本。这项工作的意义在于，它把判别式聚类（如正则化信息最大化 RIM）与生成对抗学习统一到同一框架下，为无监督和半监督分类提供了一条有理论依据、又能利用深度神经网络灵活性的新路径。
原文 arXiv:1511.06390;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1511.06390v2