GarmentGAN: Photo-realistic Adversarial Fashion Transfer
Amir Hossein Raffiee Michael Sollami Salesforce Einstein
Abstract
The garment transfer problem comprises two tasks: learning to separate a person’s body (pose, shape, color) from their clothing (garment type, shape, style) and then generating new images of the wearer dressed in arbitrary garments. We present GarmentGAN, a new algorithm that performs image-based garment transfer through generative adversarial methods. The GarmentGAN framework allows users to virtually try-on items before purchase and generalizes to various apparel types. GarmentGAN requires as input only two images, namely, a picture of the target fashion item and an image containing the customer. The output is a synthetic image wherein the customer is wearing the target apparel. In order to make the generated image look photo-realistic, we employ the use of novel generative adversarial techniques [\citenameKubo et al. 2018]. GarmentGAN improves on existing methods in the realism of generated imagery and solves various problems related to self-occlusions. Our proposed model incorporates additional information during training, utilizing both segmentation maps and body key-point information. We show qualitative and quantitative comparisons to several other networks to demonstrate th
中文速览
虚拟试衣间一直面临一个难题：如何把任意款式的衣服真实感地"穿"到顾客照片上，同时保留衣物细节（如Logo、领口形状）并应对复杂姿势和遮挡。GarmentGAN 提出了一个两阶段对抗生成框架：第一阶段用"形状迁移网络"预测穿上目标服装后的语义分割图，第二阶段先用薄板样条变换（Thin-Plate-Spline）对齐衣物形状，再由"外观迁移网络"生成最终的高分辨率彩色图像，同时引入关键点信息来专门处理手部遮挡问题。在公开时装数据集上的定量与定性对比实验表明，GarmentGAN 在图像真实感、细节保留和姿势鲁棒性上均优于现有方法。这项工作为电商虚拟试衣提供了一个无需3D数据、仅凭两张图片即可运行的实用方案，有望大幅降低消费者网购服装的决策门槛。
原文 arXiv:2003.01894;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2003.01894v1