Multiple Object Recognition with Visual Attention
Jimmy Lei Ba111Work done while at Google DeepMind. University of Toronto Volodymyr Mnih Google DeepMind Koray Kavukcuoglu Google DeepMind
Abstract
We present an attention-based model for recognizing multiple objects in images. The proposed model is a deep recurrent neural network trained with reinforcement learning to attend to the most relevant regions of the input image. We show that the model learns to both localize and recognize multiple objects despite being given only class labels during training. We evaluate the model on the challenging task of transcribing house number sequences from Google Street View images and show that it is both more accurate than the state-of-the-art convolutional networks and uses fewer parameters and less computation.
中文速览
识别街景图像中的门牌号序列一直面临两个难题：如何同时完成目标定位和字符识别，以及如何在不依赖额外标注位置信息的条件下实现端到端训练。研究者提出了一种基于注意力机制（attention）的深度循环神经网络，模型每次只截取图像中一小块区域（称为"glimpse"，类似人眼凝视），并用强化学习驱动网络自主决定下一步该看哪里，逐个识别序列中的每个数字，直到输出终止符为止。实验在谷歌街景门牌号数据集上进行，结果显示该模型在只给定字符标签（而非位置标注）的情况下自动学会了定位与识别，准确率超过当时最先进的卷积神经网络，同时参数量更少、计算量更低。这项工作的意义在于，它将目标定位与序列识别统一到一个可端到端训练的框架中，且计算开销与原始图像尺寸无关，为处理大尺寸图像中的多目标识别提供了更高效的路径。
原文 arXiv:1412.7755;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1412.7755v2