Diffusion Based Representation Learning
Sarthak Mittal Korbinian Abstreiter Stefan Bauer Bernhard Schölkopf Arash Mehrjou
Abstract
Diffusion-based methods, represented as stochastic differential equations on a continuous-time domain, have recently proven successful as non-adversarial generative models. Training such models relies on denoising score matching, which can be seen as multi-scale denoising autoencoders. Here, we augment the denoising score matching framework to enable representation learning without any supervised signal. GANs and VAEs learn representations by directly transforming latent codes to data samples. In contrast, the introduced diffusion-based representation learning relies on a new formulation of the denoising score matching objective and thus encodes the information needed for denoising. We illustrate how this difference allows for manual control of the level of details encoded in the representation. Using the same approach, we propose to learn an infinite-dimensional latent code that achieves improvements on state-of-the-art models on semi-supervised image classification. We also compare the quality of learned representations of diffusion score matching with other methods like autoencoder and contrastively trained systems through their performances on downstream tasks. Finally, we also
中文速览
扩散模型（diffusion model）虽然在图像生成上表现出色，但原始框架并没有像GAN或VAE那样自带一个编码器来学习数据的表征。这篇论文提出了"基于扩散的表征学习"（Diffusion-based Representation Learning，DRL）方法：通过在去噪得分匹配（denoising score matching）目标函数中引入一个可训练的编码器，让模型在无监督条件下从干净图像中提取潜在表征，并将其作为额外输入辅助去噪过程。关键洞察在于，作者证明了去噪得分匹配的目标函数存在一个不可消除的下界常数，而只有当编码器真正从原始数据中提炼出有效信息时，这个下界才能趋近于零——这一性质天然地驱动编码器学习有意义的表征；同时，通过调整不同噪声时间步的权重函数，可以手动控制所编码特征的粒度（粗粒度语义或细粒度细节）。实验表明，DRL在半监督图像分类上超越了当时的最优方法，并在下游任务上优于自编码器和对比学习方法，为扩散模型补齐了表征学习能力，同时还带来了更快的采样速度。
原文 arXiv:2105.14257;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2105.14257v4