StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks
Han Zhang Tao Xu Hongsheng Li Shaoting Zhang Xiaogang Wang Xiaolei Huang Dimitris Metaxas
Abstract
Synthesizing high-quality images from text descriptions is a challenging problem in computer vision and has many practical applications. Samples generated by existing text-to-image approaches can roughly reflect the meaning of the given descriptions, but they fail to contain necessary details and vivid object parts. In this paper, we propose Stacked Generative Adversarial Networks (StackGAN) to generate 256 $\times$ 256 photo-realistic images conditioned on text descriptions. We decompose the hard problem into more manageable sub-problems through a sketch-refinement process. The Stage-\@slowromancapi@ GAN sketches the primitive shape and colors of the object based on the given text description, yielding Stage-\@slowromancapi@ low-resolution images. The Stage-\@slowromancapii@ GAN takes Stage-\@slowromancapi@ results and text descriptions as inputs, and generates high-resolution images with photo-realistic details. It is able to rectify defects in Stage-\@slowromancapi@ results and add compelling details with the refinement process. To improve the diversity of the synthesized images and stabilize the training of the conditional-GAN, we introduce a novel Conditioning Augmentation tec
中文速览
文字生成高质量图像长期面临分辨率低、细节缺失的瓶颈，StackGAN（堆叠生成对抗网络）把这个难题拆成两步来解决：第一阶段的生成器根据文字描述画出物体的大致轮廓和颜色，生成低分辨率草图；第二阶段再以草图和原始文字为输入，补全细节、修正缺陷，最终输出 256×256 的逼真图像。为了让训练更稳定、生成结果更多样，研究者还提出了"条件增强"（Conditioning Augmentation）技术，通过从高斯分布中随机采样文字的潜在向量来平滑条件流形、缓解数据稀疏问题。在鸟类和花卉等基准数据集上的实验表明，StackGAN 生成的图像质量显著优于当时所有方法，首次实现了纯粹依赖文字描述就能生成照片级真实感高分辨率图像，为文字驱动的图像合成研究树立了新标杆。
原文 arXiv:1612.03242;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1612.03242v2