Sparse R-CNN: End-to-End Object Detection with Learnable Proposals
Peize Sun1∗ Rufeng Zhang2∗ Yi Jiang3∗ Tao Kong3 Chenfeng Xu4 Wei Zhan4 Masayoshi Tomizuka4 Lei Li3 Zehuan Yuan3 Changhu Wang3 Ping Luo1 1The University of Hong Kong 2Tongji University 3ByteDance AI Lab 4University of California Berkeley
Abstract
We present Sparse R-CNN, a purely sparse method for object detection in images. Existing works on object detection heavily rely on dense object candidates, such as $k$ anchor boxes pre-defined on all grids of image feature map of size $H\times W$ . In our method, however, a fixed sparse set of learned object proposals, total length of $N$ , are provided to object recognition head to perform classification and location. By eliminating $HWk$ (up to hundreds of thousands) hand-designed object candidates to $N$ (\eg100) learnable proposals, Sparse R-CNN completely avoids all efforts related to object candidates design and many-to-one label assignment. More importantly, final predictions are directly output without non-maximum suppression post-procedure. Sparse R-CNN demonstrates accuracy, run-time and training convergence performance on par with the well-established detector baselines on the challenging COCO dataset, \eg, achieving 45.0 AP in standard $3\times$ training schedule and running at 22 fps using ResNet-50 FPN model. We hope our work could inspire re-thinking the convention of dense prior in object detectors. The code is available at: https://github.com/PeizeSun/SparseR-CNN.
中文速览
目标检测领域长期依赖"密集候选框"范式——在整张特征图的每个网格上预设大量锚框或参考点，再筛选出真正的目标，这不仅带来繁琐的标签分配规则，还必须用非极大值抑制（NMS）去除重复预测。Sparse R-CNN 提出用一组数量极少（如100个）的可学习"提案框+提案特征"直接替代几十万个密集候选，每个提案特征为对应的检测头动态生成专属参数，实现真正的"稀疏输入、稀疏输出"，全程无需NMS和手工设计的候选框。在COCO基准上，该方法以ResNet-50 FPN骨干网络达到45.0 AP、22 fps，与主流密集检测器持平，同时训练收敛速度和运行效率同样具有竞争力。这项工作首次证明彻底稀疏的检测器在精度上完全可行，为重新审视目标检测中"密集先验必不可少"这一惯例提供了有力依据。
原文 arXiv:2011.12450;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2011.12450v2