The Mythos of Model Interpretability
Zachary C. Lipton
Abstract
Supervised machine learning models boast remarkable predictive capabilities. But can you trust your model? Will it work in deployment? What else can it tell you about the world? We want models to be not only good, but interpretable. And yet the task of interpretation appears underspecified. Papers provide diverse and sometimes non-overlapping motivations for interpretability, and offer myriad notions of what attributes render models interpretable. Despite this ambiguity, many papers proclaim interpretability axiomatically, absent further explanation. In this paper, we seek to refine the discourse on interpretability. First, we examine the motivations underlying interest in interpretability, finding them to be diverse and occasionally discordant. Then, we address model properties and techniques thought to confer interpretability, identifying transparency to humans and post-hoc explanations as competing notions. Throughout, we discuss the feasibility and desirability of different notions, and question the oft-made assertions that linear models are interpretable and that deep neural networks are not.
中文速览
机器学习模型的预测能力越来越强，但在医疗、司法、金融等高风险领域落地时，人们发现仅凭预测结果并不够——我们还需要知道模型为什么这么判断、能否信任它、它是否存在偏见，这就是"可解释性（interpretability）"被反复提及的原因。作者系统梳理了学术界对可解释性的各种动机与定义，发现它们彼此分散、有时甚至相互矛盾：有人说可解释性是为了建立信任，有人说是为了发现因果关系，有人说是为了确保模型在部署环境中仍然有效，还有人说是为了满足公平与法律要求。在技术层面，文章将可解释性归纳为两类截然不同的思路——模型本身对人透明（transparency）和事后解释（post-hoc explanation），并指出二者是相互竞争而非等价的概念。基于这一分析，作者对两个流行断言提出质疑：线性模型并不天然可解释，深度神经网络也并非注定是黑箱。这篇文章的价值在于它为整个领域提供了一张概念地图，帮助研究者在声称某模型"可解释"时更加精确，避免把不同目标混为一谈。
原文 arXiv:1606.03490;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1606.03490v3