Temporal Context Network for Activity Localization in Videos
Xiyang Dai11{}^{1}\quad\quad Bharat Singh11{}^{1}\quad\quad Guyue Zhang2 1University of Maryland College Park, MD xdai, bharat, Larry S. Davis11{}^{1}\qquad Yan Qiu Chen2 2Fudan University Shanghai, China guyuezhang13,
Abstract
We present a Temporal Context Network (TCN) for precise temporal localization of human activities. Similar to the Faster-RCNN architecture, proposals are placed at equal intervals in a video which span multiple temporal scales. We propose a novel representation for ranking these proposals. Since pooling features only inside a segment is not sufficient to predict activity boundaries, we construct a representation which explicitly captures context around a proposal for ranking it. For each temporal segment inside a proposal, features are uniformly sampled at a pair of scales and are input to a temporal convolutional neural network for classification. After ranking proposals, non-maximum suppression is applied and classification is performed to obtain final detections. TCN outperforms state-of-the-art methods on the ActivityNet dataset and the THUMOS14 dataset.
中文速览
人们在视频中定位活动片段（temporal activity localization）时面临一个关键难题：仅靠提案片段内部的特征，无法判断活动边界究竟在哪里。时序上下文网络（Temporal Context Network，TCN）通过在每个候选提案的当前尺度和更大尺度上同时采样特征，让模型能够"看到"片段边界之外的上下文信息，再用时序卷积网络对这对多尺度特征进行排序，筛选出高质量提案后再做分类。实验结果表明，TCN 在 ActivityNet 和 THUMOS14 两个主流数据集上均超越了当时最先进的方法，且其生成的提案还能进一步提升其他检测框架（如 CDC 网络）的性能。这项工作揭示了时序上下文对活动边界预测的关键作用，为后续视频理解研究提供了一种简洁有效的多尺度表示思路。
原文 arXiv:1708.02349;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1708.02349v1