Semi-supervised Learning with Deep Generative Models
Diederik P. Kingma∗, Danilo J. Rezende†, Shakir Mohamed†, Max Welling∗ ∗Machine Learning Group, Univ. of Amsterdam, {D.P.Kingma, †Google Deepmind, {danilor,
Abstract
The ever-increasing size of modern data sets combined with the difficulty of obtaining label information has made semi-supervised learning one of the problems of significant practical importance in modern data analysis. We revisit the approach to semi-supervised learning with generative models and develop new models that allow for effective generalisation from small labelled data sets to large unlabelled ones. Generative approaches have thus far been either inflexible, inefficient or non-scalable. We show that deep generative models and approximate Bayesian inference exploiting recent advances in variational methods can be used to provide significant improvements, making generative approaches highly competitive for semi-supervised learning.
中文速览
标注数据稀缺而无标注数据大量存在的场景下，如何让分类器充分利用无标注样本一直是个难题。研究者提出了两种深度生成模型（deep generative model）：一种先用变分自编码器把数据压缩成低维特征再训练分类器，另一种则把类别标签本身建模为潜变量，对无标注样本在推断时直接"猜"出类别，两者还可叠加成更深的随机网络。为了让这套生成模型在大规模数据上跑得动，作者引入了摊销变分推断（amortised variational inference），用神经网络作为识别网络来近似后验分布，并设计了同时优化生成参数与变分参数的随机梯度目标函数。实验结果显示，在MNIST等基准任务上，仅用100个标注样本就达到了当时的最优分类性能，同时模型还能自动将图像的"类别内容"与"书写风格"解耦，展示出生成式半监督方法在表达能力和可扩展性上的巨大潜力。
原文 arXiv:1406.5298;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1406.5298v2