Finding Action Tubes
Georgia Gkioxari UC Berkeley Jitendra Malik UC Berkeley
Abstract
We address the problem of action detection in videos. Driven by the latest progress in object detection from 2D images, we build action models using rich feature hierarchies derived from shape and kinematic cues. We incorporate appearance and motion in two ways. First, starting from image region proposals we select those that are motion salient and thus are more likely to contain the action. This leads to a significant reduction in the number of regions being processed and allows for faster computations. Second, we extract spatio-temporal feature representations to build strong classifiers using Convolutional Neural Networks. We link our predictions to produce detections consistent in time, which we call action tubes. We show that our approach outperforms other techniques in the task of action detection.
中文速览
视频中的动作不仅要被识别出来，还要被精确定位——即回答"动作在哪里发生"，而非仅仅判断"视频里有没有某个动作"。研究者借鉴图像目标检测的思路，先用选择性搜索（selective search）生成候选区域，再通过光流运动显著性过滤掉约85%的静止区域，大幅降低计算量，然后用两个卷积神经网络（CNN）分别提取外观特征和运动特征，并训练SVM分类器对每帧的候选区域进行预测，最后将各帧的检测结果在时间轴上串联成"动作管（action tube）"，得到时序一致的动作定位结果。在UCF Sports数据集上，该方法以显著优势超越所有对比方法，在交叠阈值0.6时平均AUC达41.2%，比此前最优提升87.3%；在J-HMDB数据集上，动作分类准确率也从56.6%提升至62.5%。这项工作的重要性在于，它将图像级目标检测的成熟框架成功迁移到视频动作定位任务中，证明了外观与运动双流特征的互补性，为后续视频理解研究奠定了重要基础。
原文 arXiv:1411.6031;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1411.6031v1