Underspecification Presents Challenges for Credibility in Modern Machine Learning
\nameAlexander D’Amour \nameKatherine Heller††\nameDan Moldovan††\nameBen Adlam \nameBabak Alipanahi \nameAlex Beutel \nameChristina Chen \nameJonathan Deaton \nameJacob Eisenstein \nameMatthew D. Hoffman \nameFarhad Hormozdiari \nameNeil Houlsby \nameShaobo Hou \nameGhassen Jerfel \nameAlan Karthikesalingam \nameMario Lucic \nameYian Ma \nameCory McLean \nameDiana Mincu \nameAkinori Mitani \nameAndrea Montanari \nameZachary Nado \nameVivek Natarajan \nameChristopher Nielson \nameThomas F. Osborne††\nameRajiv Raman \nameKim Ramasamy \nameRory Sayres \nameJessica Schrouff \nameMartin Seneviratne \nameShannon Sequeira \nameHarini Suresh \nameVictor Veitch \nameMax Vladymyrov \nameXuezhi Wang \nameKellie Webster \nameSteve Yadlowsky \nameTaedong Yun \nameXiaohua Zhai \nameD. Sculley These authors contributed equally to this work.This paper represents the views of the authors, and not of the VA.
Abstract
ML models often exhibit unexpectedly poor behavior when they are deployed in real-world domains. We identify underspecification as a key reason for these failures. An ML pipeline is underspecified when it can return many predictors with equivalently strong held-out performance in the training domain. Underspecification is common in modern ML pipelines, such as those based on deep learning. Predictors returned by underspecified pipelines are often treated as equivalent based on their training domain performance, but we show here that such predictors can behave very differently in deployment domains. This ambiguity can lead to instability and poor model behavior in practice, and is a distinct failure mode from previously identified issues arising from structural mismatch between training and deployment domains. We show that this problem appears in a wide variety of practical ML pipelines, using examples from computer vision, medical imaging, natural language processing, clinical risk prediction based on electronic health records, and medical genomics. Our results show the need to explicitly account for underspecification in modeling pipelines that are intended for real-world deployme
中文速览
训练时表现一样好的机器学习模型，部署到真实场景后却可能表现截然不同——这篇论文把这一现象的根源归结为"欠规范"（underspecification）：当一条ML流水线能训练出大量在验证集上性能相当的模型时，它实际上无法区分哪个模型才真正"学到了正确的东西"，最终返回哪个模型几乎是随机的。作者在计算机视觉、医学影像、自然语言处理、电子健康记录和基因组学五大领域设计了一套实验方案，只改变随机种子等与验证集性能无关的细节，结果发现不同运行产生的模型在鲁棒性、公平性和因果可靠性上存在显著差异，而这种差异正是欠规范的直接体现，而非训练域与部署域之间结构性错位所能解释的。研究结果表明，仅凭独立同分布（iid）验证集上的表现来筛选和信任模型是不够的，业界需要针对部署场景专门设计"压力测试"，并开发能够主动约束模型学到可信归纳偏置的训练方法，才能真正保证模型在现实世界中的可靠性。
原文 arXiv:2011.03395;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2011.03395v2