Dissociating language and thought in large language models
Kyle Mahowald* The University of Texas at Austin \AndAnna A. Ivanova* Georgia Institute of Technology \ANDIdan A. Blank University of California Los Angeles \AndNancy Kanwisher Massachusetts Institute of Technology \AndJoshua B. Tenenbaum Massachusetts Institute of Technology \AndEvelina Fedorenko Massachusetts Institute of Technology
Abstract
Large Language Models (LLMs) have come closest among all models to date to mastering human language, yet opinions about their linguistic and cognitive capabilities remain split. Here, we evaluate LLMs using a distinction between formal linguistic competence—knowledge of linguistic rules and patterns—and functional linguistic competence—understanding and using language in the world. We ground this distinction in human neuroscience, which has shown that formal and functional competence rely on different neural mechanisms. Although LLMs are surprisingly good at formal competence, their performance on functional competence tasks remains spotty and often requires specialized fine-tuning and/or coupling with external modules. We posit that models that use language in humanlike ways would need to master both of these competence types, which, in turn, could require the emergence of mechanisms specialized for formal linguistic competence, distinct from functional competence.
中文速览
大型语言模型（LLMs）到底"懂"语言吗？研究者提出用"形式语言能力"（掌握语言规则和统计规律）与"功能语言能力"（在真实世界中用语言达成目标）这一区分来回答这个问题，并将这一区分根植于人类神经科学——大脑中处理语言的网络与负责推理、社会认知等高级认知的网络是明确分离的。评估结果显示，LLMs在形式能力上已接近人类水平，能够处理复杂语法、句法规律等语言形式问题，但在需要调动世界知识、逻辑推理、情景理解和社会认知的功能性任务上表现参差不齐，往往需要额外的微调或外部模块辅助。这一发现意味着，单靠"预测下一个词"的训练目标足以让模型学会语言的形式，却不足以让它真正"用"语言，未来要构建真正类人的语言模型，必须同时攻克这两类能力，而不能把流利的语言输出误读为真正的理解与思考。
原文 arXiv:2301.06627;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2301.06627v3