Nested Variational Compression in Deep Gaussian Processes
James Hensman and Neil D. Lawrence
Abstract
Deep Gaussian processes provide a flexible approach to probabilistic modeling of data using either supervised or unsupervised learning. For tractable inference approximations to the marginal likelihood of the model must be made. The original approach to approximate inference in these models used variational compression to allow for approximate variational marginalization of the hidden variables leading to a lower bound on the marginal likelihood of the model (Damianou and Lawrence, 2013). In this paper we extend this idea with a nested variational compression. The resulting lower bound on the likelihood can be easily parallelised or adapted for stochastic variational inference.
中文速览
深度高斯过程（Deep Gaussian Process, DGP）通过多层函数复合，能比单层高斯过程表达更丰富的函数族，但其隐变量在层间相互依赖，导致精确推断完全不可行。本文在前人变分压缩（variational compression）方法的基础上，提出了一种"嵌套变分压缩"框架，逐层引入诱导点（inducing points）并递归地构造变分下界，从而对任意深度的模型都能高效地做近似推断。所得到的似然下界可以在数据点之间自然分解，既支持并行计算，也能直接用随机梯度下降（stochastic variational inference）进行优化，大幅降低了计算代价。这项工作让深度高斯过程在大规模数据上的实用推断成为可能，同时为理解深度概率模型与深度神经网络之间的内在联系提供了新视角。
原文 arXiv:1412.1370;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1412.1370v1