Generating Images with Perceptual Similarity Metrics based on Deep Networks
Alexey Dosovitskiy University of Freiburg, Germany Thomas Brox University of Freiburg, Germany
Abstract
Image-generating machine learning models are typically trained with loss functions based on distance in the image space. This often leads to over-smoothed results. We propose a class of loss functions, which we call deep perceptual similarity metrics (DeePSiM), that mitigate this problem. Instead of computing distances in the image space, we compute distances between image features extracted by deep neural networks. This metric better reflects perceptually similarity of images and thus leads to better results. We show three applications: autoencoder training, a modification of a variational autoencoder, and inversion of deep convolutional networks. In all cases, the generated images look sharp and resemble natural images.
中文速览
图像生成模型长期面临一个棘手问题：用像素级距离（如均方误差）作为损失函数训练时，生成的图像往往模糊、缺乏细节。作者提出了一类名为 DeePSiM 的深度感知相似度损失函数（deep perceptual similarity metrics），核心思路是不再直接比较像素，而是用预训练深度卷积网络提取的特征来衡量生成图像与目标图像之间的距离，同时结合对抗训练（adversarial training）引入自然图像先验，防止生成出感知上不自然的高频噪声图像。实验在自动编码器、变分自动编码器和 AlexNet 特征反演三个任务上均显示，该方法生成的图像纹理清晰、细节丰富，视觉质量远优于传统像素损失方案，尤其在从 AlexNet 高层激活重建图像时效果提升显著。这项工作为图像生成任务提供了一个通用且有效的损失函数框架，对后续感知损失（perceptual loss）研究影响深远。
原文 arXiv:1602.02644;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1602.02644v2