Cascaded Boundary Regression for Temporal Action Detection
Abstract
Temporal action detection in long videos is an important problem. State-of-the-art methods address this problem by applying action classifiers on sliding windows. Although sliding windows may contain an identifiable portion of the actions, they may not necessarily cover the entire action instance, which would lead to inferior performance. We adapt a two-stage temporal action detection pipeline with Cascaded Boundary Regression (CBR) model. Class-agnostic proposals and specific actions are detected respectively in the first and the second stage. CBR uses temporal coordinate regression to refine the temporal boundaries of the sliding windows. The salient aspect of the refinement process is that, inside each stage, the temporal boundaries are adjusted in a cascaded way by feeding the refined windows back to the system for further boundary refinement. We test CBR on THUMOS-14 and TVSeries, and achieve state-of-the-art performance on both datasets. The performance gain is especially remarkable under high IoU thresholds, e.g. map@tIoU=0.5 on THUMOS-14 is improved from 19.0% to 31.0%.
中文速览
视频中的动作时序定位（temporal action detection）需要精确找出每个动作片段的起止时间，但传统滑动窗口方法截取的片段往往与真实动作边界对不齐，导致检测精度不高。本文提出"级联边界回归"（Cascaded Boundary Regression, CBR）方法，在两阶段检测框架中（先生成类别无关的候选片段，再做类别判断）引入时序坐标回归，并将回归后的新边界反复送回网络进行多轮迭代修正，让模型每次都能根据新边界观察到不同内容、逐步逼近精确边界。在THUMOS-14和TVSeries两个标准测试集上，CBR均超越了当时最优方法，尤其在高重叠阈值下提升显著——THUMOS-14的mAP@tIoU=0.5从19.0%跃升至31.0%。这项工作表明，把边界回归做成级联迭代的形式是提升时序动作定位精度的有效手段，对视频理解领域具有重要参考价值。
原文 arXiv:1705.01180;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1705.01180v1