Impact of Pretraining Term Frequencies on Few-Shot Reasoning
Yasaman Razeghi Robert L. Logan IV Matt Gardner Sameer Singh
Abstract
Pretrained Language Models (LMs) have demonstrated ability to perform numerical reasoning by extrapolating from a few examples in few-shot settings. However, the extent to which this extrapolation relies on robust reasoning is unclear. In this paper, we investigate how well these models reason with terms that are less frequent in the pretraining data. In particular, we examine the correlations between the model performance on test instances and the frequency of terms from those instances in the pretraining data. We measure the strength of this correlation for a number of GPT-based language models (pretrained on the Pile dataset) on various numerical deduction tasks (e.g., arithmetic and unit conversion). Our results consistently demonstrate that models are more accurate on instances whose terms are more prevalent, in some cases above $70\%$ (absolute) more accurate on the top 10% frequent terms in comparison to the bottom 10%. Overall, although LMs exhibit strong performance at few-shot numerical reasoning tasks, our results raise the question of how much models actually generalize beyond pretraining data, and we encourage researchers to take the pretraining data into account when
中文速览
大型语言模型在少样本数学推理上看似表现亮眼，但这究竟是真正学会了推理，还是只是"背"出了预训练数据里见过的答案？研究者针对这一问题，系统考察了模型在算术运算（加法、乘法）和时间单位换算等任务上的表现，与对应数字或单位在预训练语料（Pile数据集）中的出现频率之间的关联。结果发现，模型对高频词项的准确率远高于低频词项，差距在某些任务中高达70个百分点以上——也就是说，即便同样是两位数乘法，预训练时"见过更多次"的数字组合，模型就答得更准。这一规律在GPT-Neo-1.3B、2.7B和GPT-J-6B三个不同规模的模型上均高度一致，说明当前大模型的"数学能力"很大程度上依赖预训练数据的统计记忆，而非真正的泛化推理，研究者因此呼吁在评估语言模型推理能力时，必须将预训练数据的频率因素纳入考量。
原文 arXiv:2202.07206;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2202.07206v2