An Explanation of In-context Learning as Implicit Bayesian Inference
Sang Michael Xie Stanford University、Aditi Raghunathan Stanford University \ANDPercy Liang Stanford University、Tengyu Ma Stanford University
Abstract
Large language models (LMs) such as GPT-3 have the surprising ability to do in-context learning, where the model learns to do a downstream task simply by conditioning on a prompt consisting of input-output examples. The LM learns from these examples without being explicitly pretrained to learn. Thus, it is unclear what enables in-context learning. In this paper, we study how in-context learning can emerge when pretraining documents have long-range coherence. Here, the LM must infer a latent document-level concept to generate coherent next tokens during pretraining. At test time, in-context learning occurs when the LM also infers a shared latent concept between examples in a prompt. We prove when this occurs despite a distribution mismatch between prompts and pretraining data in a setting where the pretraining distribution is a mixture of HMMs. In contrast to messy large-scale datasets used to train LMs capable of in-context learning, we generate a small-scale synthetic dataset (GINC) where Transformers and LSTMs both exhibit in-context learning111The code, data, and experiments are located on GitHub and CodaLab.. Beyond the theory, experiments on GINC exhibit large-scale real-world
中文速览
大语言模型（如GPT-3）拥有"情境学习"（in-context learning）的神奇能力——只需在提示词里给几个输入输出示例，模型就能完成新任务，但为什么会这样却一直不清楚。这篇论文用隐马尔可夫模型（Hidden Markov Model, HMM）构建了一个可控的预训练数据生成框架，证明了当预训练文档具有"长程连贯性"（即需要推断隐藏的文档级概念才能生成合理的后续内容）时，情境学习会自然涌现——模型本质上是在做贝叶斯推断，从示例中推出共享的潜在概念，再据此做预测。研究者同时构建了一个小规模合成数据集GINC来验证理论，发现Transformer和LSTM都能在其上展现情境学习，且随示例数量增多、示例长度增加，准确率稳步提升。更重要的是，这一简洁框架还复现了大模型中已观察到的真实现象——模型规模扩大能提升情境学习效果（即使预训练损失相同）、示例顺序影响结果、以及零样本有时优于少样本等——为理解情境学习的内在机制提供了坚实的理论基础。
原文 arXiv:2111.02080;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2111.02080v6