Deep Convolutional Inverse Graphics Network
Tejas D. Kulkarni*1, Will Whitney*2, Pushmeet Kohli3, Joshua B. Tenenbaum4 1,2,4Computer Science and Artificial Intelligence Laboratory, MIT 1,4Brain and Cognitive Sciences, MIT 3Microsoft Research Cambridge, UK * First two authors contributed equally to this work.
Abstract
This paper presents the Deep Convolution Inverse Graphics Network (DC-IGN), a model that learns an interpretable representation of images. This representation is disentangled with respect to transformations such as out-of-plane rotations and lighting variations. The DC-IGN model is composed of multiple layers of convolution and de-convolution operators and is trained using the Stochastic Gradient Variational Bayes (SGVB) algorithm [11]. We propose a training procedure to encourage neurons in the graphics code layer to represent a specific transformation (e.g. pose or light). Given a single input image, our model can generate new images of the same object with variations in pose and lighting. We present qualitative and quantitative results of the model’s efficacy at learning a 3D rendering engine.
中文速览
研究人员希望让神经网络像计算机图形引擎一样，自动从图像中学到"姿态、光照、形状"这类彼此独立、语义清晰的表示，而不是混在一起的黑箱特征。他们提出了深度卷积逆图形网络（DC-IGN，Deep Convolution Inverse Graphics Network），用多层卷积和反卷积构成编码器-解码器结构，并结合变分自编码器训练算法（SGVB），设计了一套特殊的小批量训练策略：每次只让一个变量（如人脸方位角、仰俯角或光源方向）在批次内变化，同时"钳制"其他潜变量的梯度，强迫网络把特定变换的信息集中到指定神经元中。实验结果表明，只给一张人脸图像，模型就能生成该人在不同姿态和光照下的新图像，潜变量与真实生成参数呈现出很强的等变性，并且在椅子等高变化数据集上同样有效。这项工作证明了无需手工设计渲染引擎、仅靠数据驱动就能让深度网络自动习得可解释的三维图形表示，为"视觉即逆向图形学"这一方向提供了实用路径。
原文 arXiv:1503.03167;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1503.03167v4