Global Self-Attention Networks for Image Recognition
Shen Zhuoran, Irwan Bello, Raviteja Vemulapalli, Xuhui Jia, Ching-Hui Chen Google Research Google Seattle, WA 98103, United States Equal contribution.Ching-Hui is now at Waymo.
Abstract
Recently, a series of works in computer vision have shown promising results on various image and video understanding tasks using self-attention. However, due to the quadratic computational and memory complexities of self-attention, these works either apply attention only to low-resolution feature maps in later stages of a deep network or restrict the receptive field of attention in each layer to a small local region. To overcome these limitations, this work introduces a new global self-attention module, referred to as the GSA module, which is efficient enough to serve as the backbone component of a deep network. This module consists of two parallel layers: a content attention layer that attends to pixels based only on their content and a positional attention layer that attends to pixels based on their spatial locations. The output of this module is the sum of the outputs of the two layers. Based on the proposed GSA module, we introduce new standalone global attention-based deep networks that use GSA modules instead of convolutions to model pixel interactions. Due to the global extent of the proposed GSA module, a GSA network has the ability to model long-range pixel interactions th
中文速览
卷积神经网络处理图像时难以捕捉远距离像素之间的依赖关系，而现有的全局自注意力（global self-attention）方案要么计算开销呈像素数量的平方级增长、只能用在网络后期的低分辨率特征图上，要么把感受野限制在局部小区域内，始终无法在整个网络中实现高效的全局建模。为此，本文提出了一种新的全局自注意力模块（GSA module），它将"基于内容的注意力层"与"基于相对位置的注意力层"并联叠加：前者用一种线性复杂度的矩阵乘法重排技巧让每个像素同时关注全图所有像素的内容，后者通过轴向分解（先列后行）以 O(N√N) 的代价引入空间位置感知，两者输出直接相加，兼顾语义相似性与空间结构信息。以 ResNet 为骨架、把卷积全部替换为 GSA 模块后构建的 GSA-ResNet，在 CIFAR-100 和 ImageNet 上均以更少的参数和计算量显著超越原版 ResNet 及多种已有的局部或全局注意力网络，证明全局自注意力完全可以作为视觉主干网络的基础算子，而不仅仅是卷积网络的辅助插件。
原文 arXiv:2010.03019;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2010.03019v2