MLP-Mixer: An all-MLP Architecture for Vision
Ilya Tolstikhin∗, Neil Houlsby∗, Alexander Kolesnikov∗, Lucas Beyer∗, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, Mario Lucic, Alexey Dosovitskiy ∗equal contribution Google Research, Brain Team {tolstikhin, neilhoulsby, akolesnikov, lbeyer, xzhai, unterthiner, jessicayung†, andstein, keysers, usz, lucic, †work done during Google AI Residency
Abstract
Convolutional Neural Networks (CNNs) are the go-to model for computer vision. Recently, attention-based networks, such as the Vision Transformer, have also become popular. In this paper we show that while convolutions and attention are both sufficient for good performance, neither of them are necessary. We present MLP-Mixer, an architecture based exclusively on multi-layer perceptrons (MLPs). MLP-Mixer contains two types of layers: one with MLPs applied independently to image patches (i.e. “mixing” the per-location features), and one with MLPs applied across patches (i.e. “mixing” spatial information). When trained on large datasets, or with modern regularization schemes, MLP-Mixer attains competitive scores on image classification benchmarks, with pre-training and inference cost comparable to state-of-the-art models. We hope that these results spark further research beyond the realms of well established CNNs and Transformers.111MLP-Mixer code will be available at https://github.com/google-research/vision_transformer
中文速览
卷积神经网络和以Vision Transformer为代表的注意力机制模型长期主导计算机视觉领域，但谷歌研究团队提出了一个颠覆性的问题：卷积和注意力机制真的是必需的吗？他们给出的答案是否定的，并为此设计了MLP-Mixer——一个完全基于多层感知机（MLP）的图像分类架构，将图像切成小块后交替使用两种MLP：一种在每个位置内部混合通道特征，另一种跨所有位置混合空间信息。实验表明，在大规模数据集（如约3亿张图片的JFT-300M）上预训练后，MLP-Mixer在ImageNet上能达到87.94%的top-1准确率，在精度与计算代价的权衡上与当前最先进的CNN和Transformer模型旗鼓相当；即便在中等规模数据上配合现代正则化技术，也能取得有竞争力的成绩。这项工作的重要意义在于：它证明了好的视觉表征并不依赖于卷积的局部归纳偏置或注意力的全局交互机制，为计算机视觉架构的探索打开了更广阔的空间。
原文 arXiv:2105.01601;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2105.01601v4