Improved StyleGAN Embedding: Where are the Good Latents?
Peihao Zhu KAUSTSaudi Arabia , Rameen Abdal KAUSTSaudi Arabia , Yipeng Qin Cardiff UniversityUK , John Femiani Miami University510 E. High StOxfordOH45056USA and Peter Wonka KAUSTSaudi Arabia
Abstract
StyleGAN is able to produce photorealistic images that are almost indistinguishable from real photos. The reverse problem of finding an embedding for a given image poses a challenge. Embeddings that reconstruct an image well are not always robust to editing operations. In this paper, we address the problem of finding an embedding that both reconstructs images and also supports image editing tasks. First, we introduce a new normalized space to analyze the diversity and the quality of the reconstructed latent codes. This space can help answer the question of where good latent codes are located in latent space. Second, we propose an improved embedding algorithm using a novel regularization method based on our analysis. Finally, we analyze the quality of different embedding algorithms. We compare our results with the current state-of-the-art methods and achieve a better trade-off between reconstruction quality and editing quality.
中文速览
为了让真实人脸照片能被精准嵌入 StyleGAN 的隐空间（latent space）并同时支持后续编辑，研究者提出了一套新的分析框架与改进算法。他们发现现有方法在"重建质量"与"编辑质量"之间存在本质矛盾：嵌入越精准，隐编码往往越偏离训练分布密集区，导致编辑后图像失真。为此，他们引入了一个经过 PCA 白化的归一化隐空间 $P_N$，在这个空间中隐编码的分布接近各向同性的球形，$L_2$ 距离等价于马氏距离（Mahalanobis distance），能直观衡量一个编码离训练分布中心有多远；基于这一洞察，他们设计了新的正则化方法，在优化嵌入时将编码拉向高密度区域，从而在几乎不损失重建精度的前提下显著改善编辑效果。实验结果表明，该方法在重建质量与人脸姿态、光照、年龄等多种编辑任务上均优于现有最优方法，为后续 GAN 图像编辑研究提供了清晰的理论基础和可复用的评测框架。
原文 arXiv:2012.09036;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2012.09036v3