gvnn: Neural Network Library for Geometric Computer Vision
Ankur Handa 11 Michael Bloesch 33 Viorica Pătrăucean 22 Simon Stent 22 John McCormac 11 Andrew Davison {brendon.mccormac13, 11
Abstract
We introduce gvnn, a neural network library in Torch aimed towards bridging the gap between classic geometric computer vision and deep learning. Inspired by the recent success of Spatial Transformer Networks, we propose several new layers which are often used as parametric transformations on the data in geometric computer vision. These layers can be inserted within a neural network much in the spirit of the original spatial transformers and allow backpropagation to enable end-to-end learning of a network involving any domain knowledge in geometric computer vision. This opens up applications in learning invariance to 3D geometric transformation for place recognition, end-to-end visual odometry, depth estimation and unsupervised learning through warping with a parametric transformation for image reconstruction error.
中文速览
传统几何计算机视觉积累了大量关于三维空间变换的先验知识，但这些知识很难直接融入深度学习框架，两者之间存在明显鸿沟。为此，作者开发了 gvnn（Geometric Vision with Neural Networks）这个 Torch 神经网络库，将 SO3/SE3/Sim3 旋转平移变换、三维网格生成、投影层、逐像素光流与深度变形等几何视觉中的核心操作，封装成可微分的神经网络层，插入卷积网络后即可通过反向传播进行端到端训练。库中还集成了多种 M-估计鲁棒损失函数，以应对真实数据中的离群点问题。实验在 RGB-D 视觉里程计任务上验证了各层的正确性，初步展示了端到端学习视觉定位的可行性。这项工作的重要意义在于，它让研究者能够把几何视觉的领域知识以模块化方式直接"嵌入"神经网络，为无监督图像重建、深度估计、场景识别等任务提供了一个统一的可学习框架。
原文 arXiv:1607.07405;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1607.07405v3