PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detection
Shaoshuai Shi Li Jiang Jiajun Deng Zhe Wang Chaoxu Guo Jianping Shi Xiaogang Wang Hongsheng Li
Abstract
3D object detection is receiving increasing attention from both industry and academia thanks to its wide applications in various fields. In this paper, we propose Point-Voxel Region-based Convolution Neural Networks (PV-RCNNs) for 3D object detection on point clouds. First, we propose a novel 3D detector, PV-RCNN, which boosts the 3D detection performance by deeply integrating the feature learning of both point-based set abstraction and voxel-based sparse convolution through two novel steps, i.e., the voxel-to-keypoint scene encoding and the keypoint-to-grid RoI feature abstraction. Second, we propose an advanced framework, PV-RCNN++, for more efficient and accurate 3D object detection. It consists of two major improvements: sectorized proposal-centric sampling for efficiently producing more representative keypoints, and VectorPool aggregation for better aggregating local point features with much less resource consumption. With these two strategies, our PV-RCNN++ is about $3\times$ faster than PV-RCNN, while also achieving better performance. The experiments demonstrate that our proposed PV-RCNN++ framework achieves state-of-the-art 3D detection performance on the large-scale and h
中文速览
自动驾驶等场景里的3D目标检测需要同时兼顾精度和速度，而现有方法要么用体素（voxel）卷积效率高但定位精度受量化误差拖累，要么用点云（point）直接处理精度高但计算量太大，两者难以兼顾。这篇论文提出了PV-RCNN和PV-RCNN++两套框架，核心思路是把基于体素的稀疏卷积（sparse convolution）和基于点的集合抽象（set abstraction）深度融合：先用体素卷积高效提取多尺度特征并生成候选框，再通过关键点把体素特征"摆渡"到精确的点位置，最终汇聚回候选框网格做精细化修正。在此基础上，PV-RCNN++进一步引入分区域的以候选框为中心的关键点采样策略和更紧凑的VectorPool局部特征聚合模块，在速度上比PV-RCNN快约3倍，内存占用也大幅降低。实验结果显示，PV-RCNN++在大规模Waymo Open Dataset上达到当时最优检测性能，并以10帧/秒的速度处理150m×150m范围内的场景，为工程落地提供了切实可行的解决方案。
原文 arXiv:2102.00463;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2102.00463v3