Revealing the Dark Secrets of BERT
Olga Kovaleva, Alexey Romanov, Anna Rogers, Anna Rumshisky Department of Computer Science University of Massachusetts Lowell Lowell, MA 01854
Abstract
BERT-based architectures currently give state-of-the-art performance on many NLP tasks, but little is known about the exact mechanisms that contribute to its success. In the current work, we focus on the interpretation of self-attention, which is one of the fundamental underlying components of BERT. Using a subset of GLUE tasks and a set of handcrafted features-of-interest, we propose the methodology and carry out a qualitative and quantitative analysis of the information encoded by the individual BERT’s heads. Our findings suggest that there is a limited set of attention patterns that are repeated across different heads, indicating the overall model overparametrization. While different heads consistently use the same attention patterns, they have varying impact on performance across different tasks. We show that manually disabling attention in certain heads leads to a performance improvement over the regular fine-tuned BERT models.
中文速览
大量NLP任务靠BERT刷出好成绩，但它内部的自注意力（self-attention）机制究竟在做什么，一直不太清楚。研究者选取GLUE中七个子任务，逐一分析BERT全部144个注意力头（attention head）的权重分布，发现这些头反复出现五种固定模式——关注特殊符号、关注前后相邻词、混合型、句内块状、以及真正随输入变化的"异质型"——说明模型存在严重的过参数化，大量头其实在做重复的事。进一步实验表明，不同头对不同任务的贡献差异显著，手动屏蔽掉某些头反而能让多个任务的性能提升最高达3.2个百分点。这项工作不仅为理解BERT的工作机制提供了系统性的分析框架，还给出了一个出乎意料却简单有效的提升方法，对后续模型压缩与优化研究有重要参考价值。
原文 arXiv:1908.08593;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1908.08593v2