GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs
Jiani Zhang1, Xingjian Shi, Junyuan Xie3, Hao Ma4, Irwin King1, Dit-Yan Yeung2 1The Chinese University of Hong Kong, Hong Kong, China, {jnzhang, 2Hong Kong University of Science and Technology, Hong Kong, China, {xshiab, 3Amazon Web Services, WA, USA, 4Microsoft Research, WA, USA, These two authors contributed equally.
Abstract
We propose a new network architecture, Gated Attention Networks (GaAN), for learning on graphs. Unlike the traditional multi-head attention mechanism, which equally consumes all attention heads, GaAN uses a convolutional sub-network to control each attention head’s importance. We demonstrate the effectiveness of GaAN on the inductive node classification problem. Moreover, with GaAN as a building block, we construct the Graph Gated Recurrent Unit (GGRU) to address the traffic speed forecasting problem. Extensive experiments on three real-world datasets show that our GaAN framework achieves state-of-the-art results on both tasks.
中文速览
图神经网络在处理社交网络分类、交通速度预测等图结构任务时，多头注意力（multi-head attention）是一种强大的聚合方式，但它对所有注意力头一视同仁，无法区分哪些头更重要。为此，研究者提出了门控注意力网络（Gated Attention Networks, GaAN），核心思路是用一个轻量卷积子网络为每个注意力头生成一个0到1之间的软门控值，从而动态调节各头的贡献权重，而非让所有头平等发声。在节点分类任务（PPI、Reddit数据集）以及以GaAN为基础模块构建的图门控循环单元（Graph Gated Recurrent Unit, GGRU）在交通速度预测任务（METR-LA数据集）上，GaAN均达到了当时最优的性能。这项工作意义在于：它提供了一种通用且计算开销极小的方式来提升图注意力机制的表达能力，同时给出了将任意图聚合器扩展为图循环神经网络的统一框架，为后续时空预测研究奠定了基础。
原文 arXiv:1803.07294;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1803.07294v1