Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation
Huiyu Wang Work done while an intern at Google.11 Yukun Zhu 22 Bradley Green 22 Hartwig Adam 22 Alan Yuille 11 Liang-Chieh Chen 22
Abstract
Convolution exploits locality for efficiency at a cost of missing long range context. Self-attention has been adopted to augment CNNs with non-local interactions. Recent works prove it possible to stack self-attention layers to obtain a fully attentional network by restricting the attention to a local region. In this paper, we attempt to remove this constraint by factorizing 2D self-attention into two 1D self-attentions. This reduces computation complexity and allows performing attention within a larger or even global region. In companion, we also propose a position-sensitive self-attention design. Combining both yields our position-sensitive axial-attention layer, a novel building block that one could stack to form axial-attention models for image classification and dense prediction. We demonstrate the effectiveness of our model on four large-scale datasets. In particular, our model outperforms all existing stand-alone self-attention models on ImageNet. Our Axial-DeepLabhttps://github.com/csrhddlam/axial-deeplab improves 2.8% PQ over bottom-up state-of-the-art on COCO test-dev. This previous state-of-the-art is attained by our small variant that is 3.8 $\times$ parameter-efficient
中文速览
卷积神经网络（CNN）依赖局部感受野，天然难以捕捉图像中的长距离依赖关系，而直接在全图做二维自注意力（self-attention）又计算量极大。本文提出将二维自注意力分解为沿高度轴和宽度轴依次执行的两个一维自注意力，即"轴向注意力"（axial-attention），同时引入同时依赖查询（query）和键（key）的位置敏感设计，让模型在保持全局感受野的同时大幅降低计算复杂度。基于这一模块，作者构建了 Axial-ResNet 用于图像分类，并在其之上衍生出 Axial-DeepLab 用于全景分割、实例分割和语义分割，在 ImageNet 上超越了所有现有独立自注意力模型，在 COCO test-dev 上比自底向上方法的先前最优结果提升了 2.8% PQ，同时在 Mapillary Vistas 和 Cityscapes 上也达到了新的最优性能。这项工作首次实现了具有大范围乃至全局感受野的纯注意力网络，为在视觉任务中用自注意力替代卷积提供了既高效又精准的新路径。
原文 arXiv:2003.07853;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2003.07853v2