Directly Training Joint Energy-Based Models for Conditional Synthesis and Calibrated Prediction of Multi-Attribute Data
Jacob Kelly Richard Zemel Will Grathwohl
Abstract
Multi-attribute classification generalizes classification, presenting new challenges for making accurate predictions and quantifying uncertainty. We build upon recent work and show that architectures for multi-attribute prediction can be reinterpreted as energy-based models (EBMs). While existing EBM approaches achieve strong discriminative performance, they are unable to generate samples conditioned on novel attribute combinations. We propose a simple extension which expands the capabilities of EBMs to generating accurate conditional samples. Our approach, combined with newly developed techniques in energy-based model training, allows us to directly maximize the likelihood of data and labels under the unnormalized joint distribution. We evaluate our proposed approach on high-dimensional image data with high-dimensional binary attribute labels. We find our models are capable of both accurate, calibrated predictions and high-quality conditional synthesis of novel attribute combinations.
中文速览
多属性分类（multi-attribute classification）要求模型同时预测每张图片的多个标签，还要做到预测准确且不确定性估计可靠，这比普通单标签分类难得多。研究者在联合能量模型（Joint Energy-Based Model，JEM）框架上做了改进，提出了一种叫 Gibbs-JEM 的方法：设计了一个"Langevin-Within-Gibbs"采样器，让模型直接对数据和标签的联合分布做最大似然训练，而不是像以往那样把联合分布拆开分别优化。在鞋子图像（UT Zappos50K）和人脸图像（CelebA）上的实验表明，Gibbs-JEM 不仅保持了与监督基线相当的分类准确率，还显著提升了预测校准性（calibration）和异常检测能力，更关键的是，它能直接生成以稀有属性组合乃至训练集中从未出现过的属性组合为条件的高质量图像。这项工作的意义在于，它打通了判别式学习与生成式学习之间的壁垒，用一个统一模型同时做好"预测"和"生成"两件事，为处理标签稀缺、属性组合爆炸等现实场景提供了新思路。
原文 arXiv:2108.04227;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2108.04227v1