Coupled Generative Adversarial Networks
Ming-Yu Liu Mitsubishi Electric Research Labs (MERL),、Oncel Tuzel Mitsubishi Electric Research Labs (MERL),
Abstract
We propose coupled generative adversarial network (CoGAN) for learning a joint distribution of multi-domain images. In contrast to the existing approaches, which require tuples of corresponding images in different domains in the training set, CoGAN can learn a joint distribution without any tuple of corresponding images. It can learn a joint distribution with just samples drawn from the marginal distributions. This is achieved by enforcing a weight-sharing constraint that limits the network capacity and favors a joint distribution solution over a product of marginal distributions one. We apply CoGAN to several joint distribution learning tasks, including learning a joint distribution of color and depth images, and learning a joint distribution of face images with different attributes. For each task it successfully learns the joint distribution without any tuple of corresponding images. We also demonstrate its applications to domain adaptation and image transformation.
中文速览
联合生成对抗网络（Coupled GAN, CoGAN）要解决的核心难题是：如何在没有任何配对图像（例如同一场景的彩色图与深度图）的情况下，仅凭各域独立采样就学到多域图像的联合分布。方法的关键在于让两个并行的GAN共享生成器前几层（负责解码高层语义）和判别器后几层的权重，迫使两个网络以相同方式理解高层概念，再各自用独立的低层网络将共享表示渲染成不同域的图像。实验结果表明，CoGAN在数字图像、边缘图、深度图、人脸属性等多个任务上均能无监督地生成语义一致的跨域配对图像，还可直接用于无监督域适应和图像转换。这项工作打破了以往方法对配对训练数据的依赖，大幅降低了多域图像学习的数据收集门槛，为后续图像翻译研究（如CycleGAN等）奠定了重要基础。
原文 arXiv:1606.07536;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1606.07536v2