Model Patching: Closing the Subgroup Performance Gap with Data Augmentation
Karan Goel Albert Gu Yixuan Li Christopher Ré
Abstract
Classifiers in machine learning are often brittle when deployed. Particularly concerning are models with inconsistent performance on specific subgroups of a class, e.g., exhibiting disparities in skin cancer classification in the presence or absence of a spurious bandage. To mitigate these performance differences, we introduce model patching, a two-stage framework for improving robustness that encourages the model to be invariant to subgroup differences, and focus on class information shared by subgroups. Model patching first models subgroup features within a class and learns semantic transformations between them, and then trains a classifier with data augmentations that deliberately manipulate subgroup features. We instantiate model patching with CAMEL, which (1) uses a CycleGAN to learn the intra-class, inter-subgroup augmentations, and (2) balances subgroup performance using a theoretically-motivated subgroup consistency regularizer, accompanied by a new robust objective. We demonstrate CAMEL’s effectiveness on 3 benchmark datasets, with reductions in robust error of up to 33% relative to the best baseline. Lastly, CAMEL successfully patches a model that fails due to spurious fe
中文速览
皮肤癌分类器有时会把"有绷带"这类无关特征当成判断良性的线索，导致不同子群体之间性能差异悬殊。为此，论文提出"模型修补"（Model Patching）两阶段框架：第一阶段用 CycleGAN 自动学习同类别不同子群之间的语义转换（比如给图片加上或去掉绷带），第二阶段把这些转换当作数据增强，配合一个新的子群鲁棒目标函数和有理论保证的一致性正则化器来训练分类器，迫使模型对子群差异免疫。在 MNIST、CelebA、Waterbirds 三个基准上，该方法将最坏情况误差最多降低 33%，在真实皮肤癌数据集 ISIC 上鲁棒准确率提升 11.7%。这项工作的价值在于，它首次将"学习子群间变换"与"理论可解释的一致性约束"结合起来，为机器学习公平性和鲁棒性问题提供了一套无需人工设计增强规则的自动化解决方案。
原文 arXiv:2008.06775;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2008.06775v1