Learnable Explicit Density
Chin-Wei Huang Ahmed Touati Laurent Dinh Michal Drozdzal Mohammad Havaei Laurent Charlin Aaron Courville
Abstract
In this paper, we study two aspects of the variational autoencoder (VAE): the prior distribution over the latent variables and its corresponding posterior. First, we decompose the learning of VAEs into layerwise density estimation, and argue that having a flexible prior is beneficial to both sample generation and inference. Second, we analyze the family of inverse autoregressive flows (inverse AF) and show that with further improvement, inverse AF could be used as universal approximation to any complicated posterior. Our analysis results in a unified approach to parameterizing a VAE, without the need to restrict ourselves to use factorial Gaussians in the latent real space.
中文速览
变分自编码器（VAE）在图像生成上效果不如自回归模型或GAN，根本原因在于两个薄弱环节：先验分布（prior）过于简单，以及后验近似（approximate posterior）不够灵活。本文从理论上证明，优化VAE的变分下界在极限条件下会驱使先验与聚合后验对齐，因此让先验可学习（用Real NVP等可逆变换来参数化）能直接改善生成采样质量；同时，作者还证明了逆自回归流（Inverse Autoregressive Flow，IAF）在神经网络参数化下是任意后验分布的万能近似器，从理论上为用IAF改进变分推断提供了保证。在此基础上，作者提出将可学习的显式密度模型同时用于先验和后验，并在二值MNIST上验证了该方法——灵活先验带来的提升尤为显著，在此之上再引入灵活后验可进一步降低负对数似然。这项工作填补了VAE理论与实践之间的空白，为设计更强大的深度生成模型提供了有据可查的方向。
原文 arXiv:1710.02248;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1710.02248v1