DA-RNN: Semantic Mapping with Data Associated Recurrent Neural Networks
Yu Xiang and Dieter Fox Paul G. Allen School of Computer Science、Engineering University of Washington {yuxiang,
Abstract
3D scene understanding is important for robots to interact with the 3D world in a meaningful way. Most previous works on 3D scene understanding focus on recognizing geometrical or semantic properties of a scene independently. In this work, we introduce Data Associated Recurrent Neural Networks (DA-RNNs), a novel framework for joint 3D scene mapping and semantic labeling. DA-RNNs use a new recurrent neural network architecture for semantic labeling on RGB-D videos. The output of the network is integrated with mapping techniques such as KinectFusion in order to inject semantic information into the reconstructed 3D scene. Experiments conducted on real world and synthetic RGB-D videos demonstrate the superior performance of our method.
中文速览
机器人要真正理解周围环境，既需要知道三维空间的几何结构，也需要知道每个区域属于哪类物体，而现有方法往往把这两件事分开处理、联系松散。为此，研究者提出了数据关联循环神经网络（Data Associated Recurrent Neural Networks，DA-RNNs），其核心创新是一种新型循环单元（DA-RU）：它借助KinectFusion等三维重建算法提供的像素级数据关联，让网络在相邻帧之间以空间一致的方式传递和积累语义特征，而不是像传统RNN那样假定帧间连接固定不变。网络对每一帧RGB-D图像输出逐像素的语义标签，这些标签再被融合进KinectFusion的三维体素地图，最终得到几何与语义兼备的稠密三维场景。在真实RGB-D场景数据集和基于ShapeNet生成的合成数据集上，DA-RNNs的语义标注精度均优于已有方法。这项工作的价值在于它将视频帧分析与三维建图紧密耦合为一个可端到端训练的框架，为机器人感知和人机交互中的三维语义理解提供了更高效、更准确的解决思路。
原文 arXiv:1703.03098;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1703.03098v2