The Unreliability of Explanations in Few-shot Prompting for Textual Reasoning
Xi Ye Greg Durrett Department of Computer Science The University of Texas at Austin
Abstract
Does prompting a large language model (LLM) like GPT-3 with explanations improve in-context learning? We study this question on two NLP tasks that involve reasoning over text, namely question answering and natural language inference. We test the performance of four LLMs on three textual reasoning datasets using prompts that include explanations in multiple different styles. For these tasks, we find that including explanations in the prompts for OPT, GPT-3 (davinci), and InstructGPT (text-davinci-001) only yields small to moderate accuracy improvements over standard few-show learning. However, text-davinci-002 is able to benefit more substantially.
中文速览
让大语言模型（LLM）在推理时先给出解释、再作预测，真的能提升它的表现吗？研究者在问答和自然语言推理两类任务上测试了四款模型，发现对大多数模型而言，把解释加入提示词（prompt）只带来微小甚至负面的效果，仅有text-davinci-002能从中显著获益，而这一结论与此前在数学、符号推理等程序化任务上发现的"解释大幅提升性能"相矛盾。更令人意外的是，模型生成的解释往往并不真实可靠——即便在精心设计的合成数据集上，模型也会捏造与输入矛盾的内容，尽管它的解释通常在表面上与自己的预测保持一致。不过研究者发现，解释的可靠程度本身可以作为预测是否正确的信号：解释越真实、越逻辑自洽，对应预测越可能是对的；据此他们训练了一个校准器，利用自动提取的解释质量特征对模型输出进行事后修正，在所有数据集上均实现了性能提升。这项工作揭示了将解释引入上下文学习（in-context learning）的真实局限，并为如何借助解释进行可靠性评估提供了实用思路。
原文 arXiv:2205.03401;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2205.03401v2