Nonparametric Estimation of Multi-View Latent Variable Models
Le Song Email: College of Computing, Georgia Institute of Technology Animashree Anandkumar Email: EECS, University of California Irvine Bo Dai Email: College of Computing, Georgia Institute of Technology Bo Xie Email: College of Computing, Georgia Institute of Technology
Abstract
Spectral methods have greatly advanced the estimation of latent variable models, generating a sequence of novel and efficient algorithms with strong theoretical guarantees. However, current spectral algorithms are largely restricted to mixtures of discrete or Gaussian distributions. In this paper, we propose a kernel method for learning multi-view latent variable models, allowing each mixture component to be nonparametric. The key idea of the method is to embed the joint distribution of a multi-view latent variable into a reproducing kernel Hilbert space, and then the latent parameters are recovered using a robust tensor power method. We establish that the sample complexity for the proposed method is quadratic in the number of latent components and is a low order polynomial in the other relevant parameters. Thus, our non-parametric tensor approach to learning latent variable models enjoys good sample and computational efficiencies. Moreover, the non-parametric tensor power method compares favorably to EM algorithm and other existing spectral algorithms in our experiments.
中文速览
混合模型中每个分量的分布形态往往未知，传统谱方法（spectral methods）只能处理离散分布或高斯分布，换了别的分布就束手无策。本文提出一种基于核嵌入（kernel embedding）的非参数张量方法：把多视角潜变量模型（multi-view latent variable model）的联合分布映射到再生核希尔伯特空间（reproducing kernel Hilbert space，RKHS），利用二阶协方差算子做核奇异值分解提取低秩结构，再对三阶协方差算子执行鲁棒张量幂方法（tensor power method）来恢复潜在参数，整个过程无需对混合分量的分布形式做任何假设。理论分析表明，该方法的样本复杂度关于潜在分量数量呈二次增长，关于其他参数仅为低次多项式，兼顾了计算效率和统计效率。实验结果显示，当模型假设正确时本方法与EM算法及现有谱方法性能相当，而当分布假设不匹配时本方法大幅优于竞争方法，为学习潜变量模型提供了更通用、更稳健的统一框架。
原文 arXiv:1311.3287;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1311.3287v2