Variational Deep Embedding: An Unsupervised and Generative Approach to Clustering
Zhuxi Jiang1, Yin Zheng2, Huachun Tan1, Bangsheng Tang3, Hanning Zhou3 1Beijing Institute of Technology, Beijing, China 2Tencent AI Lab, Shenzhen, China 3Hulu LLC., Beijing, China {zjiang,
Abstract
Clustering is among the most fundamental tasks in machine learning and artificial intelligence. In this paper, we propose Variational Deep Embedding (VaDE), a novel unsupervised generative clustering approach within the framework of Variational Auto-Encoder (VAE). Specifically, VaDE models the data generative procedure with a Gaussian Mixture Model (GMM) and a deep neural network (DNN): 1) the GMM picks a cluster; 2) from which a latent embedding is generated; 3) then the DNN decodes the latent embedding into an observable. Inference in VaDE is done in a variational way: a different DNN is used to encode observables to latent embeddings, so that the evidence lower bound (ELBO) can be optimized using the Stochastic Gradient Variational Bayes (SGVB) estimator and the reparameterization trick. Quantitative comparisons with strong baselines are included in this paper, and experimental results show that VaDE significantly outperforms the state-of-the-art clustering methods on $5$ benchmarks from various modalities. Moreover, by VaDE’s generative nature, we show its capability of generating highly realistic samples for any specified cluster, without using supervised information during tr
中文速览
把变分自编码器（VAE）和高斯混合模型（GMM）结合起来做无监督聚类，是一个自然的想法，但如何让两者真正融为一体、共同优化，并非易事。VaDE（Variational Deep Embedding）的做法是：用GMM先选一个簇，再从该簇对应的高斯分布中采样出潜在向量，最后由深度神经网络将潜在向量解码为可观测数据，整个生成过程通过变分推断和重参数化技巧端到端地联合优化，使潜在空间天然呈现出多峰的混合高斯结构。在MNIST、Reuters、HHAR等五个来自不同模态的基准数据集上，VaDE的聚类精度大幅超越DEC、AAE等当时最强的基线方法。更重要的是，得益于其生成式本质，VaDE无需任何标注信息，就能为指定的任意簇生成高度逼真的样本，证明它学到的表示真实捕捉了数据的内在结构，而不仅仅是为聚类任务"硬拟合"出来的特征。
原文 arXiv:1611.05148;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1611.05148v3