Hierarchical Implicit Models and Likelihood-Free Variational Inference
Dustin Tran Columbia University、Rajesh Ranganath Princeton University、David M. Blei Columbia University
Abstract
Implicit probabilistic models are a flexible class of models defined by a simulation process for data. They form the basis for theories which encompass our understanding of the physical world. Despite this fundamental nature, the use of implicit models remains limited due to challenges in specifying complex latent structure in them, and in performing inferences in such models with large data sets. In this paper, we first introduce hierarchical implicit models (hims). hims combine the idea of implicit densities with hierarchical Bayesian modeling, thereby defining models via simulators of data with rich hidden structure. Next, we develop likelihood-free variational inference (lfvi), a scalable variational inference algorithm for hims. Key to lfvi is specifying a variational family that is also implicit. This matches the model’s flexibility and allows for accurate approximation of the posterior. We demonstrate diverse applications: a large-scale physical simulator for predator-prey populations in ecology; a Bayesian generative adversarial network for discrete data; and a deep implicit model for text generation.
中文速览
真实世界中很多系统（生态模拟、物理仿真、生成对抗网络等）天然是"隐式"的——你能从中采样生成数据，却写不出一个解析的概率密度函数，这使得对它们做贝叶斯推断极为困难。本文提出了两项配套工作：一是"层次隐式模型"（Hierarchical Implicit Models, HIMs），把经典贝叶斯层次建模与隐式模拟器结合起来，允许模型同时拥有局部潜变量、全局参数和任意复杂的生成过程；二是"无似然变分推断"（Likelihood-Free Variational Inference, LFVI），利用密度比估计（把判别器当作对数密度比的代理）绕开似然不可算的障碍，同时让变分族本身也可以是隐式的，从而以随机优化的方式扩展到百万级数据。在三个实验中——10万条时间序列的捕食者-猎物生态模拟、针对离散数据的贝叶斯GAN分类任务、以及带噪声隐层的深度隐式文本生成——LFVI都展现出传统ABC方法无法触及的规模与精度，为把模拟器驱动的科学模型纳入大规模贝叶斯分析开辟了一条可行路径。
原文 arXiv:1702.08896;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1702.08896v3