Large Language Models Can Be Easily Distracted by Irrelevant Context
Freda Shi Xinyun Chen Kanishka Misra Nathan Scales David Dohan Ed Chi Nathanael Schärli Denny Zhou
Abstract
Large language models have achieved impressive performance on various natural language processing tasks. However, so far they have been evaluated primarily on benchmarks where all information in the input context is relevant for solving the task. In this work, we investigate the distractibility of large language models, i.e., how the model problem-solving accuracy can be influenced by irrelevant context. In particular, we introduce Grade-School Math with Irrelevant Context (GSM-IC), an arithmetic reasoning dataset with irrelevant information in the problem description. We use this benchmark to measure the distractibility of cutting-edge prompting techniques for large language models, and find that the model performance is dramatically decreased when irrelevant information is included. We also identify several approaches for mitigating this deficiency, such as decoding with self-consistency and adding to the prompt an instruction that tells the language model to ignore the irrelevant information.111Dataset is available at https://github.com/google-research-datasets/GSM-IC.
中文速览
大型语言模型（LLM）在数学推理等任务上表现亮眼，但现有评测基准几乎都只给模型提供"干净"的题目，而现实世界中的问题往往夹杂着与解题无关的信息。为此，研究者构建了 GSM-IC 数据集——在小学数学题（GSM8K）的原始描述中插入一句无关信息——用来专门衡量主流提示技术（链式思维、最小到最多提示等）抵抗"干扰"的能力。实验发现，一旦加入无关句子，所有被测提示方法的准确率都大幅下滑，其中能对所有干扰类型保持稳定正确的题目不超过 18%；而采用自洽解码（self-consistency，即对同一题采样多次取多数答案）、在示例中加入含无关信息的样本，或在提示前加一句"请忽略无关信息"的指令，都能有效缓解这一问题。这项工作揭示了当前顶尖语言模型在上下文理解和信息筛选上的根本性缺陷，提示未来评测模型推理能力时不能只看它能否解难题，还必须考察它面对冗余信息时是否容易跑偏。
原文 arXiv:2302.00093;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2302.00093v3