Many-Shot In-Context Learning
Rishabh Agarwal Contributed equally Avi Singh Contributed equally Lei M. Zhang Key contribution Bernd Bohnet Key contribution Luis Rosias Key contribution Stephanie C.Y. Chan Key contribution Biao Zhang Key contribution Ankesh Anand Zaheer Abbas Azade Nova John D. Co-Reyes Eric Chu Feryal Behbahani Aleksandra Faust Hugo Larochelle
Abstract
Large language models (LLMs) excel at few-shot in-context learning (ICL) – learning from a few input-output examples (“shots”) provided in context at inference, without any weight updates. Newly expanded context windows allow us to investigate ICL with hundreds or thousands of examples – the many-shot regime. Going from few-shot to many-shot, we observe significant performance gains across a wide variety of generative and discriminative tasks. While promising, many-shot ICL can be bottlenecked by the available amount of human-generated outputs. To mitigate this limitation, we explore two settings: (1) “Reinforced ICL” that uses model-generated chain-of-thought rationales in place of human rationales, and (2) “Unsupervised ICL” where we remove rationales altogether, and prompt the model only with domain-specific inputs. We find that both Reinforced and Unsupervised ICL can be effective in the many-shot regime, particularly on complex reasoning tasks. Furthermore, we demonstrate that, unlike few-shot learning, many-shot learning is effective at overriding pretraining biases, can learn high-dimensional functions with numerical inputs, and performs comparably to fine-tuning. Finally, w
中文速览
大语言模型（LLM）一直依赖少量示例来做上下文学习（in-context learning，ICL），核心瓶颈在于上下文窗口太短。随着 Gemini 1.5 Pro 将上下文扩展到百万 token，研究者系统测试了在数百乃至数千个示例下 ICL 的表现，发现跨越数学推理、机器翻译、摘要、规划等多类任务时，"多样本学习"（many-shot ICL）相比少样本有显著提升。针对人工标注数据稀缺的瓶颈，论文还提出两种替代方案：用模型自生成并经正确性过滤的推理链替代人工标注（强化 ICL），以及干脆去掉答案、只提供问题输入（无监督 ICL），两者在复杂推理任务上均能超过少样本人工示例基线。此外研究还揭示了多样本 ICL 的几个重要特性：它能纠正预训练阶段形成的偏见、媲美全量微调的效果、处理纯数值输入的高维预测任务，而且下一个 token 的预测损失并不能可靠预测 ICL 在推理任务上的实际表现。这项工作表明，当上下文足够长时，ICL 有望成为微调的高效替代方案，大幅降低对人工标注数据的依赖。
原文 arXiv:2404.11018;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2404.11018v3