Image Transformer
Niki Parmar Ashish Vaswani Jakob Uszkoreit Łukasz Kaiser Noam Shazeer Alexander Ku Dustin Tran
Abstract
Image generation has been successfully cast as an autoregressive sequence generation or transformation problem. Recent work has shown that self-attention is an effective way of modeling textual sequences. In this work, we generalize a recently proposed model architecture based on self-attention, the Transformer, to a sequence modeling formulation of image generation with a tractable likelihood. By restricting the self-attention mechanism to attend to local neighborhoods we significantly increase the size of images the model can process in practice, despite maintaining significantly larger receptive fields per layer than typical convolutional neural networks. While conceptually simple, our generative models significantly outperform the current state of the art in image generation on ImageNet, improving the best published negative log-likelihood on ImageNet from 3.83 to 3.77. We also present results on image super-resolution with a large magnification ratio, applying an encoder-decoder configuration of our architecture. In a human evaluation study, we find that images generated by our super-resolution model fool human observers three times more often than the previous state of the ar
中文速览
让图像生成模型既能"看得远"又能算得快，是个长期未解决的难题：卷积网络感受野太小，循环网络又太慢。研究者把自然语言处理中大放异彩的 Transformer（基于自注意力机制）引入图像生成，提出了 Image Transformer，并通过把自注意力限制在每个像素的局部邻域内，大幅降低了计算量，使模型在保持远超卷积网络感受野的同时仍可高效并行训练。在标准 ImageNet 图像生成任务上，该模型将最优负对数似然从 3.83 降至 3.77，刷新了当时的最佳记录；在 4 倍超分辨率任务中，人类评测者将其生成图像误认为真实图像的频率是此前最优方法的三倍。这项工作证明了纯注意力架构在高分辨率视觉生成中的可行性，为后续视觉大模型的发展奠定了重要基础。
原文 arXiv:1802.05751;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1802.05751v3