The Variational Gaussian Process
Dustin Tran Harvard University \ANDRajesh Ranganath Princeton University \ANDDavid M. Blei Columbia University
Abstract
Variational inference is a powerful tool for approximate inference, and it has been recently applied for representation learning with deep generative models. We develop the variational Gaussian process (vgp), a Bayesian nonparametric variational family, which adapts its shape to match complex posterior distributions. The vgp generates approximate posterior samples by generating latent inputs and warping them through random non-linear mappings; the distribution over random mappings is learned during inference, enabling the transformed outputs to adapt to varying complexity. We prove a universal approximation theorem for the vgp, demonstrating its representative power for learning any model. For inference we present a variational objective inspired by auto-encoders and perform black box inference over a wide class of models. The vgp achieves new state-of-the-art results for unsupervised learning, inferring models such as the deep latent Gaussian model and the recently proposed DRAW.
中文速览
变分推断（variational inference）是贝叶斯机器学习中用于近似复杂后验分布的核心工具，但传统的均场（mean-field）方法因假设各潜变量相互独立而表达能力有限。这篇论文提出了一种新的变分族——变分高斯过程（variational Gaussian process，VGP），它先从简单分布中采样潜在输入，再通过由高斯过程（Gaussian process）生成的随机非线性映射将其扭曲变换，从而产生能逼近任意复杂后验的近似样本；其中"变分数据"（一组输入输出对）作为可优化参数，引导映射向真实后验集中。论文从理论上证明了VGP的普适逼近定理，即在温和条件下，VGP可以逼近任何具有连续分位函数的后验分布，并设计了受自编码器（auto-encoder）启发的黑箱推断算法，无需对目标模型做特殊假设。在无监督学习基准测试上，VGP应用于深度隐变量高斯模型（deep latent Gaussian model）和注意力生成模型DRAW，取得了当时最优的结果，表明该方法既有坚实的理论保证，又在实际深度生成模型中具有显著的性能优势。
原文 arXiv:1511.06499;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1511.06499v4