Video-based Person Re-identification via 3D Convolutional Networks and Non-local Attention
Xingyu Liao111Authors contributed equally. 1 , Lingxiao He111Authors contributed equally. 2 , Zhouwang Yang1 Chi Zhang3 1University of Science and Technology of China, Hefei, P.R.China 2University of Chinese Academy of Sciences, Beijing, P.R.China 3 Megvii Inc.(Face++)
Abstract
Video-based person re-identification (ReID) is a challenging problem, where some video tracks of people across non-overlapping cameras are available for matching. Feature aggregation from a video track is a key step for video-based person ReID. Many existing methods tackle this problem by average/maximum temporal pooling or RNNs with attention. However, these methods cannot deal with temporal dependency and spatial misalignment problems at the same time. We are inspired by video action recognition that involves the identification of different actions from video tracks. Firstly, we use 3D convolutions on video volume, instead of using 2D convolutions across frames, to extract spatial and temporal features simultaneously. Secondly, we use a non-local block to tackle the misalignment problem and capture spatial-temporal long-range dependencies. As a result, the network can learn useful spatial-temporal information as a weighted sum of the features in all space and temporal positions in the input feature map. Experimental results on three datasets show that our framework outperforms state-of-the-art approaches by a large margin on multiple metrics.
中文速览
视频行人重识别（Video-based Person Re-ID）面临的核心难题是：如何从多摄像头拍摄的视频片段中同时处理好时序依赖、空间错位和遮挡等问题，而现有的时序池化或循环神经网络方案只能顾此失彼。为此，研究者借鉴视频动作识别的思路，用三维卷积网络（3D CNN，具体为在Kinetics上预训练的膨胀式3D ResNet-50）直接从视频体积中联合提取空间和时序特征，同时嵌入非局部注意力模块（Non-local Block）来捕捉全局时空长程依赖，从而在一个端到端框架中同时解决时序建模与空间错位两大问题。在MARS、iLIDS-VID、PRID-2011三个主流数据集上的实验表明，该方法在多项指标上均大幅超越当时的最优方法。这项工作的意义在于，它将3D卷积与非局部自注意力机制首次系统地引入视频行人重识别，为后续利用时空整体建模来提升识别精度提供了清晰的思路和有效的基准。
原文 arXiv:1807.05073;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1807.05073v3