Do pretrained Transformers Learn In-Context by Gradient Descent?
Lingfeng Shen Aayush Mishra Daniel Khashabi
Abstract
The emergence of In-Context Learning (ICL) in LLMs remains a remarkable phenomenon that is partially understood. To explain ICL, recent studies have created theoretical connections to Gradient Descent (GD). We ask, do such connections hold up in actual pre-trained language models? We highlight the limiting assumptions in prior works that make their setup considerably different from the practical setup in which language models are trained. For example, their experimental verification uses ICL objective (training models explicitly for ICL), which differs from the emergent ICL in the wild. Furthermore, the theoretical hand-constructed weights used in these studies have properties that don’t match those of real LLMs. We also look for evidence in real models. We observe that ICL and GD have different sensitivity to the order in which they observe demonstrations. Finally, we probe and compare the ICL vs. GD hypothesis in a natural setting. We conduct comprehensive empirical analyses on language models pre-trained on natural data (LLaMa-7B). Our comparisons of three performance metrics highlight the inconsistent behavior of ICL and GD as a function of various factors such as datasets, mod
中文速览
大语言模型（LLM）能在不更新参数的情况下仅凭几个示例就学会新任务，这种"上下文学习"（In-Context Learning，ICL）能力令人惊叹，而近年来一批理论研究试图用梯度下降（Gradient Descent，GD）来解释它，声称两者在算法层面等价。本文对这一等价假说提出了系统性质疑：作者指出，现有理论研究存在严重的前提偏差——它们要么用专门为ICL设计的目标函数来训练模型，而非真实的自监督预训练；要么依赖手工构造的、与真实LLM权重特性相去甚远的参数矩阵，因此根本无法代表现实中涌现出ICL能力的语言模型。在实证层面，作者以LLaMA-7B为对象进行了大量实验，发现ICL和GD在示例顺序敏感性、多种性能指标以及对输出概率分布的影响上均表现出明显差异，且这种差异随数据集、模型规模和示例数量的变化呈现出不一致的规律。这些发现表明，"ICL等价于GD"仍是一个悬而未决的假说，提示学界需要在更贴近真实预训练场景的条件下重新审视这一问题，避免将受限实验条件下的结论过度推广到实际部署的大模型。
原文 arXiv:2310.08540;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2310.08540v5