Do Prompt-Based Models Really Understand the Meaning of Their Prompts?
Albert Webson Department of Computer Science, Brown University Department of Philosophy, Brown University Ellie Pavlick Department of Computer Science, Brown University
Abstract
Recently, a boom of papers has shown extraordinary progress in zero-shot and few-shot learning with various prompt-based models. It is commonly argued that prompts help models to learn faster in the same way that humans learn faster when provided with task instructions expressed in natural language. In this study, we experiment with over 30 prompt templates manually written for natural language inference (NLI). We find that models learn just as fast with many prompts that are intentionally irrelevant or even pathologically misleading as they do with instructively “good” prompts. Further, such patterns hold even for models as large as 175 billion parameters (Brown et al., 2020) as well as the recently proposed instruction-tuned models which are trained on hundreds of prompts (Sanh et al., 2021). That is, instruction-tuned models often produce good predictions with irrelevant and misleading prompts even at zero shots. In sum, notwithstanding prompt-based models’ impressive improvement, we find evidence of serious limitations that question the degree to which such improvement is derived from models understanding task instructions in ways analogous to humans’ use of task instructions.
中文速览
大量研究声称，给语言模型加上自然语言提示词（prompt）能显著提升零样本和小样本学习能力，理由是这些提示就像人类任务说明一样帮助模型"理解"任务。然而，这项研究在自然语言推理（NLI）任务上测试了30多种手动编写的提示模板，发现无论模板内容是"好的"任务说明、故意误导的假说明，还是与任务毫不相关的闲话，模型的学习速度几乎没有区别——即便是拥有1750亿参数的GPT-3，以及专门在数百条提示上做过指令微调（instruction tuning）的T0模型，也表现出同样的规律。更有趣的是，指令微调模型在零样本下对无关甚至错误的提示也能给出不错的预测，说明它们对提示语义"过于不敏感"；而模型真正在意的，是预测时用哪个目标词（如"yes"/"no"），而非模板本身的含义。这一发现对提示方法的本质提出了严重质疑：现有模型的性能提升，可能并不像人们所想的那样源于对任务说明的真正理解。
原文 arXiv:2109.01247;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2109.01247v2