Density estimation using Real NVP
Laurent Dinh Montreal Institute for Learning Algorithms University of Montreal Montreal, QC H3T1J4 \ANDJascha Sohl-Dickstein Google Brain、Samy Bengio Google Brain Work was done when author was at Google Brain.
Abstract
Unsupervised learning of probabilistic models is a central yet challenging problem in machine learning. Specifically, designing models with tractable learning, sampling, inference and evaluation is crucial in solving this task. We extend the space of such models using real-valued non-volume preserving (real NVP) transformations, a set of powerful, stably invertible, and learnable transformations, resulting in an unsupervised learning algorithm with exact log-likelihood computation, exact and efficient sampling, exact and efficient inference of latent variables, and an interpretable latent space. We demonstrate its ability to model natural images on four datasets through sampling, log-likelihood evaluation, and latent variable manipulations.
中文速览
真实值非体积保留变换（real NVP）针对的是无监督生成模型长期面临的两难困境——要么采样方便但概率密度难以精确计算，要么推断准确但生成效率低下。该方法通过设计一种特殊的可逆变换（仿射耦合层），巧妙地让雅可比矩阵保持三角形结构，从而把行列式计算量压缩到线性时间内，并在此基础上堆叠多尺度架构，同时享有精确对数似然计算、高效精确采样、精确潜变量推断三大优势。在 CIFAR-10、ImageNet、LSUN 等四个自然图像数据集上的实验表明，real NVP 能生成视觉质量可观的样本，并能在有意义的潜空间中做插值与语义操作。这项工作的重要性在于，它第一次把"可精确计算密度的流模型"推广到大规模真实图像上，为后续标准化流（normalizing flow）研究奠定了基础，同时也为半监督学习和图像编辑等下游应用提供了一个兼具表达力与可解释性的生成框架。
原文 arXiv:1605.08803;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1605.08803v3