Robust Representation Learning via Perceptual Similarity Metrics
Saeid Asgari Taghanaki Kristy Choi Amir Khasahmadi Anirudh Goyal
Abstract
A fundamental challenge in artificial intelligence is learning useful representations of data that yield good performance on a downstream task, without overfitting to spurious input features. Extracting such task-relevant predictive information is particularly difficult for real-world datasets. In this work, we propose Contrastive Input Morphing (CIM), a representation learning framework that learns input-space transformations of the data to mitigate the effect of irrelevant input features on downstream performance. Our method leverages a perceptual similarity metric via a triplet loss to ensure that the transformation preserves task-relevant information. Empirically, we demonstrate the efficacy of our approach on tasks which typically suffer from the presence of spurious correlations: classification with nuisance information, out-of-distribution generalization, and preservation of subgroup accuracies. We additionally show that CIM is complementary to other mutual information-based representation learning techniques, and demonstrate that it improves the performance of variational information bottleneck (VIB) when used together.
中文速览
真实世界的机器学习模型常常会抓住数据里的"伪相关"特征（比如图片背景颜色）来做预测，导致在分布外数据或少数子群上表现糟糕。为了解决这个问题，作者提出了"对比输入变形"（Contrastive Input Morphing，CIM）框架：用一个卷积自编码器作为变换网络，在输入图像上学出一张像素级掩码，把与任务无关的信息直接从输入里"抹掉"；训练信号来自一个以结构相似度指标（SSIM）为度量的三元组损失，让变换后的输入在视觉统计上向同类样本靠拢、远离异类样本，从而用类间共享信息作为伪相关的代理信号。在带干扰背景的彩色MNIST、背景挑战、VLCS域泛化、CelebA和Waterbirds子群准确率等五个基准上，CIM及其与变分信息瓶颈（VIB）的组合版本均优于经验风险最小化基线和同类无监督方法，且无需任何关于伪相关属性的额外标注。这项工作首次将SSIM引入对比学习框架，为在输入空间做有损变换以增强模型鲁棒性提供了一条简洁且可即插即用的新路径。
原文 arXiv:2106.06620;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2106.06620v1