JoJoGAN: One Shot Face Stylization
Min Jin Chong D.A. Forsyth
Abstract
A style mapper applies some fixed style to its input images (so, for example, taking faces to cartoons). This paper describes a simple procedure – JoJoGAN – to learn a style mapper from a single example of the style. JoJoGAN uses a GAN inversion procedure and StyleGAN’s style-mixing property to produce a substantial paired dataset from a single example style. The paired dataset is then used to fine-tune a StyleGAN. An image can then be style mapped by GAN-inversion followed by the fine-tuned StyleGAN. JoJoGAN needs just one reference and as little as $30$ seconds of training time. JoJoGAN can use extreme style references (say, animal faces) successfully. Furthermore, one can control what aspects of the style are used and how much of the style is applied. Qualitative and quantitative evaluation show that JoJoGAN produces high quality high resolution images that vastly outperform the current state-of-the-art.
中文速览
单张卡通或动漫风格参考图能否直接训练出一个高质量的人脸风格化模型，这正是 JoJoGAN 要解决的问题。它的核心思路是：先用 GAN 反演（GAN inversion）把那张参考图映射回 StyleGAN 的隐空间，再利用 StyleGAN 的风格混合（style mixing）特性，从这一个样本出发自动扩充出大量配对训练数据，然后用这些数据配合基于判别器特征的感知损失对 StyleGAN 进行微调，最终只需约 30 秒训练就能得到一个可用的风格映射器。实验表明，JoJoGAN 不仅能处理人脸卡通化等常见场景，还能应对动物脸这类极端参考图，并支持对风格强度和风格细节进行灵活控制，在定性和定量评估上都大幅超越现有最优方法。这项工作的意义在于，它让普通用户只凭一张图片、不到一分钟的时间，就能获得高分辨率、高保真度的个性化风格化效果，极大降低了创作门槛。
原文 arXiv:2112.11641;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2112.11641v4