3D Semantic Segmentation with Submanifold Sparse Convolutional Networks
Benjamin Graham , Martin Engelcke , Laurens van der Maaten {benjamingraham, Facebook AI ResearchUniversity of Oxford, intern at Facebook AI Research
Abstract
Convolutional networks are the de-facto standard for analyzing spatio-temporal data such as images, videos, and 3D shapes. Whilst some of this data is naturally dense (e.g., photos), many other data sources are inherently sparse. Examples include 3D point clouds that were obtained using a LiDAR scanner or RGB-D camera. Standard “dense” implementations of convolutional networks are very inefficient when applied on such sparse data. We introduce new sparse convolutional operations that are designed to process spatially-sparse data more efficiently, and use them to develop spatially-sparse convolutional networks. We demonstrate the strong performance of the resulting models, called submanifold sparse convolutional networks (SSCNs), on two tasks involving semantic segmentation of 3D point clouds. In particular, our models outperform all prior state-of-the-art on the test set of a recent semantic segmentation competition.
中文速览
稀疏的3D点云数据（比如激光雷达扫描得到的场景）用传统卷积网络处理极其浪费——大量计算都花在空白区域上，而且每经过一层卷积，稀疏结构还会不断"膨胀"变密。为此，作者提出了一种新的卷积算子——流形子空间稀疏卷积（Submanifold Sparse Convolution，SSC），其核心思想是：只有当感受野中心点本身是活跃点时，输出才被激活，从而让稀疏结构在网络的每一层都保持不变，彻底解决了"子流形膨胀"问题。在此基础上，作者构建了一套完整的稀疏卷积网络框架（SSCN），配合类似U-Net的多尺度结构实现高效的逐点语义分割，计算和内存开销都与活跃点数量成正比而非与整个网格体积挂钩。在3D点云语义分割的公开竞赛测试集上，SSCN超越了所有已有方案，证明这一方法在保持极高效率的同时也能达到顶尖精度，对LiDAR感知、RGB-D视频理解等高维稀疏数据处理具有重要实用价值。
原文 arXiv:1711.10275;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1711.10275v1