Identifying through Flows for Recovering Latent Representations
Shen Li Institute of Data Science、NUS Graduate School for Integrative Sciences and Engineering National University of Singapore \ANDBryan Hooi、Gim Hee Lee Department of Computer Science, National University of Singapore
Abstract
Identifiability, or recovery of the true latent representations from which the observed data originates, is de facto a fundamental goal of representation learning. Yet, most deep generative models do not address the question of identifiability, and thus fail to deliver on the promise of the recovery of the true latent sources that generate the observations. Recent work proposed identifiable generative modelling using variational autoencoders (iVAE) with a theory of identifiability. Due to the intractablity of KL divergence between variational approximate posterior and the true posterior, however, iVAE has to maximize the evidence lower bound (ELBO) of the marginal likelihood, leading to suboptimal solutions in both theory and practice. In contrast, we propose an identifiable framework for estimating latent representations using a flow-based model (iFlow). Our approach directly maximizes the marginal likelihood, allowing for theoretical guarantees on identifiability, thereby dispensing with variational approximations. We derive its optimization objective in analytical form, making it possible to train iFlow in an end-to-end manner. Simulations on synthetic data validate the correctn
中文速览
真正从数据中还原出生成它的潜在因素（可辨识性，identifiability），一直是表征学习的核心难题。此前的 iVAE 虽然给出了可辨识性理论，但因为变分自编码器框架中 KL 散度难以精确计算，只能优化证据下界（ELBO），导致理论和实验结果都不够理想。本文提出 iFlow，把可辨识性理论与归一化流（normalizing flow）结合起来——由于归一化流天然支持精确推断和精确似然计算，iFlow 可以直接最大化条件边缘似然，绕开了变分近似的误差，并推导出解析形式的优化目标，实现端到端训练。在合成数据上的实验验证了 iFlow 能更准确地恢复真实潜在表征，性能明显优于 iVAE 等现有方法，为可辨识深度生成模型提供了更坚实的理论保障和更好的实际效果。
原文 arXiv:1909.12555;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1909.12555v4