Syntactic Perturbations Reveal Representational Correlates of Hierarchical Phrase Structure in Pretrained Language Models
Matteo Alleman† Jonathan Mamou‡ Miguel A Del Rio◇◇{}^{\mathbin{\Diamond}} Hanlin Tang‡ Yoon Kim⋆,◇,∙⋆◇∙{}^{\star,\mathbin{\Diamond},\bullet} SueYeon Chung†,◇,∙†◇∙{}^{{\dagger},\mathbin{\Diamond},\bullet} †Columbia University ‡Intel Labs ⋆MIT-IBM Watson AI ◇◇{}^{\mathbin{\Diamond}}Massachusetts Institute of Technology
Abstract
While vector-based language representations from pretrained language models have set a new standard for many NLP tasks, there is not yet a complete accounting of their inner workings. In particular, it is not entirely clear what aspects of sentence-level syntax are captured by these representations, nor how (if at all) they are built along the stacked layers of the network. In this paper, we aim to address such questions with a general class of interventional, input perturbation-based analyses of representations from pretrained language models. Importing from computational and cognitive neuroscience the notion of representational invariance, we perform a series of probes designed to test the sensitivity of these representations to several kinds of structure in sentences. Each probe involves swapping words in a sentence and comparing the representations from perturbed sentences against the original. We experiment with three different perturbations: (1) random permutations of $n$ -grams of varying width, to test the scale at which a representation is sensitive to word position; (2) swapping of two spans which do or do not form a syntactic phrase, to test sensitivity to global phrase
中文速览
大型预训练语言模型（如BERT）的内部表示究竟学到了哪些句法结构，一直是个谜。研究者借鉴认知神经科学中"表征不变性"的思路，设计了三类受控输入扰动实验：打乱不同大小的n-gram词序、交换句子中的短语（分有效句法短语和无效对照组两种）、以及交换相邻词对（分跨短语边界和不跨边界两种），通过测量扰动前后各层表征的偏移量来推断模型对不同句法结构的敏感程度。实验结果表明，Transformer从浅层到深层逐步建立起对更大范围词序的敏感性，而且层级短语结构在这一过程中起到了关键作用——跨越短语边界的词序改动造成的表征偏移，随两词在句法树上的距离增大而增大，且这种相关性随层数加深而增强。这项工作证明了输入扰动法可以作为监督探针之外的有效补充手段，为理解深度语言模型如何编码复杂层级结构提供了新的分析视角。
原文 arXiv:2104.07578;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2104.07578v1