Temporal Pyramid Pooling Based Convolutional Neural Network for Action Recognition
Peng Wang, Yuanzhouhan Cao, Chunhua Shen, Lingqiao Liu, and Heng Tao Shen P. Wang’s contribution was made when visiting the University of Adelaide. The first two authors made equal contributions to this work.P. Wang and H. T. Shen are with School of Information Technology and Electrical Engineering, The University of Queensland, Australia (email: Cao, C. Shen, and L. Liu are with School of Computer Science, The University of Adelaide, Australia (email: {yuanzhouhan.cao, chunhua.shen, Shen is also with Australian Centre for Robotic Vision, Australia.
Abstract
Encouraged by the success of Convolutional Neural Networks (CNNs) in image classification, recently much effort is spent on applying CNNs to video based action recognition problems. One challenge is that video contains a varying number of frames which is incompatible to the standard input format of CNNs. Existing methods handle this issue either by directly sampling a fixed number of frames or bypassing this issue by introducing a 3D convolutional layer which conducts convolution in spatial-temporal domain.
中文速览
针对视频动作识别（video-based action recognition）中帧数不固定、难以直接输入标准卷积神经网络（CNN）的痛点，研究者提出了一种新型网络结构，核心是引入编码层（encoding layer）与时序金字塔池化层（temporal pyramid pooling），将任意帧数的视频统一转换为固定长度的视频级表示，同时还通过特征拼接层将外观信息与运动信息融合。与固定帧采样相比，该方法不会遗漏关键帧；与需要百万级视频训练数据的3D卷积方法相比，该方法可直接借助在ImageNet上预训练的图像CNN来初始化参数，只需小规模数据集即可完成训练。在Hollywood2和HMDB51两个标准数据集上的实验表明，该方法以少得多的训练数据达到了超越当时最优方法的识别精度，为视频理解领域提供了一条兼顾性能与数据效率的新思路。
原文 arXiv:1503.01224;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1503.01224v2