MaskGAN: Towards Diverse and Interactive Facial Image Manipulation
Cheng-Han Lee1 Ziwei Liu2 Lingyun Wu1 Ping Luo3 1SenseTime Research 2The Chinese University of Hong Kong 3The University of Hong Kong
Abstract
Facial image manipulation has achieved great progress in recent years. However, previous methods either operate on a predefined set of face attributes or leave users little freedom to interactively manipulate images. To overcome these drawbacks, we propose a novel framework termed MaskGAN, enabling diverse and interactive face manipulation. Our key insight is that semantic masks serve as a suitable intermediate representation for flexible face manipulation with fidelity preservation. MaskGAN has two main components: 1) Dense Mapping Network (DMN) and 2) Editing Behavior Simulated Training (EBST). Specifically, DMN learns style mapping between a free-form user modified mask and a target image, enabling diverse generation results. EBST models the user editing behavior on the source mask, making the overall framework more robust to various manipulated inputs. Specifically, it introduces dual-editing consistency as the auxiliary supervision signal. To facilitate extensive studies, we construct a large-scale high-resolution face dataset with fine-grained mask annotations named CelebAMask-HQ. MaskGAN is comprehensively evaluated on two challenging tasks: attribute transfer and style copy
中文速览
人脸编辑长期面临两个痛点：要么只能修改预设的少数属性（如"微笑""年龄"），要么无法让用户自由地、精细地指定想要改动的局部区域。MaskGAN 的核心思路是把语义分割掩码（semantic mask）当作人脸编辑的中间桥梁——用户只需涂改掩码上的某个区域（比如拉大眼睛、换个发型轮廓），系统就能据此生成逼真的新人脸。为此，论文提出了两个关键模块：稠密映射网络（Dense Mapping Network, DMN）通过空间感知风格编码器，把目标人脸的细粒度风格迁移到用户修改后的掩码对应位置；编辑行为模拟训练（Editing Behavior Simulated Training, EBST）则在训练时用 MaskVAE 在掩码流形上生成微小扰动样本，再借助 Alpha Blender 保持一致性，让模型对各种用户涂改行为更加鲁棒。论文还配套构建了大规模高分辨率数据集 CelebAMask-HQ（逾 3 万张 512×512 人脸，含 19 类精细掩码标注），在属性迁移和风格复制两项任务上均超越现有方法。这项工作的意义在于，它把"画掩码"变成了一种直观、灵活的人脸编辑交互方式，同时开放的数据集也为后续研究提供了重要的基准资源。
原文 arXiv:1907.11922;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1907.11922v2