Neural Photo Editing with Introspective Adversarial Networks
Andrew Brock, Theodore Lim,、J.M. Ritchie School of Engineering and Physical Sciences Heriot-Watt University Edinburgh, UK {ajb5, t.lim, \AndNick Weston Renishaw plc Research Ave, North Edinburgh, UK
Abstract
The increasingly photorealistic sample quality of generative image models suggests their feasibility in applications beyond image generation. We present the Neural Photo Editor, an interface that leverages the power of generative neural networks to make large, semantically coherent changes to existing images. To tackle the challenge of achieving accurate reconstructions without loss of feature quality, we introduce the Introspective Adversarial Network, a novel hybridization of the VAE and GAN. Our model efficiently captures long-range dependencies through use of a computational block based on weight-shared dilated convolutions, and improves generalization performance with Orthogonal Regularization, a novel weight regularization method. We validate our contributions on CelebA, SVHN, and CIFAR-100, and produce samples and reconstructions with high visual fidelity.
中文速览
照片编辑通常需要逐像素操作且依赖专业技能，而现有的生成模型（VAE和GAN）在重建精度与图像质量之间存在难以调和的矛盾。作者提出了一种名为"神经照片编辑器"（Neural Photo Editor）的交互界面，让用户只需用"上下文画笔"在图像上涂抹，系统便会自动在隐空间中梯度下降，从而实现发型、肤色、表情等大范围、语义连贯的修改，并通过一个基于修改幅度的自适应遮罩（mask）将生成模型的编辑结果无缝迁移到真实照片上。为支撑这一界面，作者还提出了"内省对抗网络"（Introspective Adversarial Network，IAN）——将VAE的推断机制与GAN的判别器合并为同一网络，同时引入基于权重共享扩张卷积的多尺度卷积块（MDC）和正交正则化（Orthogonal Regularization），在不显著增加参数量的情况下提升感受野与泛化能力。在CelebA、SVHN、CIFAR等基准上的实验表明，IAN在图像采样、重建和半监督分类上均达到有竞争力的水平，证明了将生成模型作为通用图像编辑工具的可行性。
原文 arXiv:1609.07093;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1609.07093v3