SODA: Bottleneck Diffusion Models for Representation Learning
Drew A. Hudson⋆⋆{}^{\star}start_FLOATSUPERSCRIPT ⋆ end_FLOATSUPERSCRIPT Daniel Zoran Mateusz Malinowski Andrew K. Lampinen James L. McClelland Loic Matthey Felix Hill Alexander Lerchner
Abstract
We introduce SODA, a self-supervised diffusion model, designed for representation learning. The model incorporates an image encoder, which distills a source view into a compact representation, that, in turn, guides the generation of related novel views. We show that by imposing a tight bottleneck between the encoder and a denoising decoder, and leveraging novel view synthesis as a self-supervised objective, we can turn diffusion models into strong representation learners, capable of capturing visual semantics in an unsupervised manner. To the best of our knowledge, SODA is the first diffusion model to succeed at ImageNet linear-probe classification, and, at the same time, it accomplishes reconstruction, editing and synthesis tasks across a wide range of datasets. Further investigation reveals the disentangled nature of its emergent latent space, that serves as an effective interface to control and manipulate the produced images. All in all, we aim to shed light on the exciting and promising potential of diffusion models, not only for image generation, but also for learning rich and robust representations. See our website at soda-diffusion.github.io.
中文速览
扩散模型（diffusion model）过去主要被当作图像生成工具，却很少有人研究它能否同时学到有用的视觉表征——SODA 就是为填补这一空白而设计的自监督模型。它的核心思路是：用一个图像编码器把"源视角"图像压缩成紧凑的隐向量，再让扩散解码器以该向量为条件去重建"目标视角"图像，以新视角合成作为自监督学习信号；关键在于编码器与解码器之间设置了严格的信息瓶颈，迫使隐向量只保留高层语义，而非逐像素细节。实验表明，SODA 是首个在 ImageNet 线性探测（linear-probe）分类任务上取得强竞争力结果的扩散模型，同时在图像重建、编辑和多数据集新视角生成上也表现出色，其隐空间还呈现出自然解耦的特性，可按语义维度精确操控生成结果。这项工作证明扩散模型不只是"画图工具"，也能成为强大的无监督视觉表征学习框架，为两类研究方向的融合打开了新的可能。
原文 arXiv:2311.17901;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2311.17901v1