Synthesizing Programs for Images using Reinforced Adversarial Learning
Yaroslav Ganin Tejas Kulkarni Igor Babuschkin S. M. Ali Eslami Oriol Vinyals
Abstract
Advances in deep generative networks have led to impressive results in recent years. Nevertheless, such models can often waste their capacity on the minutiae of datasets, presumably due to weak inductive biases in their decoders. This is where graphics engines may come in handy since they abstract away low-level details and represent images as high-level programs. Current methods that combine deep learning and renderers are limited by hand-crafted likelihood or distance functions, a need for large amounts of supervision, or difficulties in scaling their inference algorithms to richer datasets. To mitigate these issues, we present SPIRAL, an adversarially trained agent that generates a program which is executed by a graphics engine to interpret and sample images. The goal of this agent is to fool a discriminator network that distinguishes between real and rendered data, trained with a distributed reinforcement learning setup without any supervision. A surprising finding is that using the discriminator’s output as a reward signal is the key to allow the agent to make meaningful progress at matching the desired output rendering. To the best of our knowledge, this is the first demonstr
中文速览
人类天生擅长从图像中恢复结构化表示，比如把手写字符分解成笔画，但让机器做到这一点却一直很难——主要障碍在于图形渲染器不可微分、离散符号序列难以优化，以及缺乏大量标注数据。SPIRAL 提出了一种全新思路：用一个循环神经网络作为"智能体"来生成绘图指令序列，交给图形渲染引擎执行后产生图像，再用一个判别器来区分生成图像与真实图像，将判别器的输出作为强化学习的奖励信号，整个系统无需任何人工标注即可端到端训练。实验表明，在 MNIST、Omniglot、CelebA 人脸以及合成 3D 场景等多个数据集上，SPIRAL 都能自动学会用笔触或场景描述来重建图像，且一个关键发现是：用判别器输出作为奖励远比直接优化像素误差更有效。这是首个无监督、对抗式的逆图形渲染智能体，为将深度学习与图形引擎结合、从真实数据中发现可解释结构提供了一条可扩展的新路径。
原文 arXiv:1804.01118;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1804.01118v1