What Context Features Can Transformer Language Models Use?
Joe O’Connor Jacob Andreas Massachusetts Institute of Technology
Abstract
Transformer-based language models benefit from conditioning on contexts of hundreds to thousands of previous tokens. What aspects of these contexts contribute to accurate model prediction? We describe a series of experiments that measure usable information by selectively ablating lexical and structural information in transformer language models trained on English Wikipedia. In both mid- and long-range contexts, we find that several extremely destructive context manipulations—including shuffling word order within sentences and deleting all words other than nouns—remove less than 15% of the usable information. Our results suggest that long contexts, but not their detailed syntactic and propositional content, are important for the low perplexity of current transformer language models.111Code for all experiments in this paper is available at https://github.com/lingo-mit/context-ablations.
中文速览
Transformer语言模型在预测下一个词时会利用数百乃至数千个历史词的上下文，但这些上下文究竟哪些部分真正起作用，一直缺乏清晰的认识。研究者借助"可用信息"（V-information）框架，通过在训练阶段对上下文做各种受控破坏——包括打乱词序、只保留名词、删除功能词等——来测量每种破坏会损失多少模型可用的预测信息。实验结果相当出人意料：即便将句子内部词序完全打乱、或只保留名词而删去其余所有词，损失的可用信息也不超过15%；真正对模型帮助最大的是局部词共现统计和内容词本身，而非精细的句法结构或命题语义内容。这一发现说明，当前Transformer语言模型的低困惑度主要来自"有多少上下文"而非"上下文写了什么"，提示未来研究应着力探索更高效的上下文表示方式，而不仅仅是堆叠更长的文本。
原文 arXiv:2106.08367;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2106.08367v1