CNN-SLAM: Real-time dense monocular SLAM with learned depth prediction
Keisuke Tateno 1,2, Federico Tombari∗1, Iro Laina1, Nassir Navab1,3 {tateno, tombari, laina, 1 CAMP - TU Munich 2 Canon Inc. 3 Johns Hopkins University Munich, Germany Tokyo, Japan Baltimore, US The first two authors contribute equally to this paper.
Abstract
Given the recent advances in depth prediction from Convolutional Neural Networks (CNNs), this paper investigates how predicted depth maps from a deep neural network can be deployed for accurate and dense monocular reconstruction. We propose a method where CNN-predicted dense depth maps are naturally fused together with depth measurements obtained from direct monocular SLAM. Our fusion scheme privileges depth prediction in image locations where monocular SLAM approaches tend to fail, e.g. along low-textured regions, and vice-versa. We demonstrate the use of depth prediction for estimating the absolute scale of the reconstruction, hence overcoming one of the major limitations of monocular SLAM. Finally, we propose a framework to efficiently fuse semantic labels, obtained from a single frame, with dense SLAM, yielding semantically coherent scene reconstruction from a single view. Evaluation results on two benchmark datasets show the robustness and accuracy of our approach.
中文速览
单目相机虽然廉价普及，却长期面临两大痛点：重建结果缺乏绝对尺度（不知道场景到底有多大），以及在低纹理区域或纯旋转运动时容易丢失跟踪。研究者提出将深度卷积神经网络（CNN）预测的稠密深度图与直接法单目SLAM（同步定位与建图）的深度估计融合起来——CNN预测负责提供全局绝对尺度和低纹理区域的深度，SLAM的小基线立体匹配则在梯度丰富的边缘区域对CNN预测进行精细化修正，两者各取所长。在此基础上，作者还引入语义分割网络，将逐像素的语义标签与稠密重建结果融合，首次在纯单目场景下实现了语义一致的三维重建。在两个公开基准数据集上的实验表明，该方法在位姿估计精度和重建质量上均优于现有单目SLAM方法，且整个系统可在GPU（CNN推断）与CPU（深度优化）并行运行下达到实时速度，对实际部署中从未见过的新场景也具有良好的泛化能力。
原文 arXiv:1704.03489;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1704.03489v1