When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Data
Peter Hase Mohit Bansal
Abstract
Many methods now exist for conditioning model outputs on task instructions, retrieved documents, and user-provided explanations and feedback. Rather than relying solely on examples of task inputs and outputs, these approaches use valuable additional data for improving model correctness and aligning learned models with human priors. Meanwhile, a growing body of evidence suggests that some language models can (1) store a large amount of knowledge in their parameters, and (2) perform inference over tasks in textual inputs at test time. These results raise the possibility that, for some tasks, humans cannot explain to a model any more about the task than it already knows or could infer on its own. In this paper, we study the circumstances under which explanations of individual data points can (or cannot) improve modeling performance. In order to carefully control important properties of the data and explanations, we introduce a synthetic dataset for experiments, and we also make use of three existing datasets with explanations: e-SNLI, TACRED, and SemEval. We first give a formal framework for the available modeling approaches, in which explanation data can be used as model inputs, as t
中文速览
大型语言模型已经能从参数里调用海量知识、并在推理时自行"猜"出任务规则，这使得"给模型附上自然语言解释（explanation）真的有用吗"成为一个悬而未决的问题。作者系统梳理了解释数据作为训练目标、模型输入、正则先验三种用法，并最终聚焦于最有潜力的一种——把历史解释检索出来直接拼入当前输入。为严格控制变量，他们专门设计了一个合成数据集，并在 e-SNLI、TACRED、SemEval 三个真实数据集上做对比：在合成任务上，基于检索的方法准确率超过 95%，而不用解释的基线仅有 65% 以下；但在三个真实数据集上，检索解释却没带来任何提升。通过分析失败案例，作者归纳出六项"解释起效"的前提条件，为社区今后判断"什么时候值得收集和使用解释数据"提供了可操作的诊断框架。
原文 arXiv:2102.02201;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2102.02201v2