A Multiscale Visualization of Attention in the Transformer Model
Jesse Vig Palo Alto Research Center 3333 Coyote Hill Road Palo Alto, CA 94304
Abstract
The Transformer is a sequence model that forgoes traditional recurrent architectures in favor of a fully attention-based approach. Besides improving performance, an advantage of using attention is that it can also help to interpret a model by showing how the model assigns weight to different input elements. However, the multi-layer, multi-head attention mechanism in the Transformer model can be difficult to decipher. To make the model more accessible, we introduce an open-source tool that visualizes attention at multiple scales, each of which provides a unique perspective on the attention mechanism. We demonstrate the tool on BERT and OpenAI GPT-2 and present three example use cases: detecting model bias, locating relevant attention heads, and linking neurons to model behavior.
中文速览
Transformer模型在BERT、GPT-2等大型语言模型中大放异彩，但其多层、多头注意力机制（multi-head attention）究竟"在看什么"，对研究者来说就像一个黑盒。作者开发了一款开源可视化工具BertViz，从三个层次拆解注意力：注意力头视图让用户看清单个头的连接模式，模型视图以缩略图矩阵一览所有层和头的全局分布，神经元视图则深入到查询向量（query）和键向量（key）的逐维乘积，揭示注意力分数是如何从底层神经元计算出来的。借助这三种视角，作者展示了三个实际用例——识别GPT-2在代词指代上的性别偏见、快速定位对释义检测任务有用的跨句注意力头、以及追踪造成距离衰减注意力模式的具体神经元。这一工具让研究者无需深挖代码就能直观理解和审查Transformer的内部机制，对模型调试、偏见检测和可解释性研究都具有实用价值。
原文 arXiv:1906.05714;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1906.05714v1