Evaluating Explanations: How much do explanations from the teacher aid students?
Danish Pruthi1 Rachit Bansal2 Bhuwan Dhingra3 Livio Baldini Soares3 Michael Collins3 Zachary C. Lipton1 Graham Neubig1 William W. Cohen3 1 Carnegie Mellon University 2 Delhi Technological University 3 Google Research {ddanish, zlipton, {bdhingra, liviobs, mjcollins, Part of this work was done at Google.
Abstract
While many methods purport to explain predictions by highlighting salient features, what aims these explanations serve and how they ought to be evaluated often go unstated. In this work, we introduce a framework to quantify the value of explanations via the accuracy gains that they confer on a student model trained to simulate a teacher model. Crucially, the explanations are available to the student during training, but are not available at test time. Compared to prior proposals, our approach is less easily gamed, enabling principled, automatic, model-agnostic evaluation of attributions. Using our framework, we compare numerous attribution methods for text classification and question answering, and observe quantitative differences that are consistent (to a moderate to high degree) across different student model architectures and learning strategies.111Code for the evaluation protocol: https://github.com/danishpruthi/evaluating-explanations
中文速览
现有的特征归因（feature attribution）方法五花八门，却缺乏公认的量化评估标准，大多只靠人工肉眼检查几个样例来"验证"好坏。这篇论文提出了一套"师生框架"：让教师模型（teacher）对训练样本给出预测标签和归因解释，再让学生模型（student）利用这些解释学习模拟教师的决策，最终只凭原始输入（不借助解释）在测试集上预测教师的输出——解释让学生模拟得越准，说明解释质量越高。这套框架的关键设计是测试时学生看不到解释，从根本上堵住了"解释直接泄露答案"的漏洞，也避免了以往方案容易被trivial策略刷高分的问题。作者在文本分类和问答两类任务上系统比较了七种主流归因方法，发现注意力机制解释和积分梯度（integrated gradients）表现最好，普通梯度显著图和LIME最差，且该排名在不同学生架构和学习策略下具有中到高度的一致性。这项工作为自动、可扩展、模型无关地评估解释方法提供了原则性基础，有助于推动可解释AI领域走向更严格的量化比较。
原文 arXiv:2012.00893;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2012.00893v2