Colorful Image Colorization
Richard Zhang Phillip Isola Alexei A. Efros
Abstract
Given a grayscale photograph as input, this paper attacks the problem of hallucinating a plausible color version of the photograph. This problem is clearly underconstrained, so previous approaches have either relied on significant user interaction or resulted in desaturated colorizations. We propose a fully automatic approach that produces vibrant and realistic colorizations. We embrace the underlying uncertainty of the problem by posing it as a classification task and use class-rebalancing at training time to increase the diversity of colors in the result. The system is implemented as a feed-forward pass in a CNN at test time and is trained on over a million color images. We evaluate our algorithm using a “colorization Turing test,” asking human participants to choose between a generated and ground truth color image. Our method successfully fools humans on 32% of the trials, significantly higher than previous methods. Moreover, we show that colorization can be a powerful pretext task for self-supervised feature learning, acting as a cross-channel encoder. This approach results in state-of-the-art performance on several feature learning benchmarks.
中文速览
给黑白照片自动上色听起来简单，实则因为颜色信息大量缺失而极度欠定——同一张灰度图对应的合理彩色方案可能有很多种。研究者把这个问题重新定义为分类任务，让卷积神经网络（CNN）预测每个像素在量化颜色空间上的概率分布，而不是直接回归出一个颜色值，从而避免了传统均方误差损失导致结果偏灰、饱和度低的问题。为了让模型不偏袒那些在自然图像中大量出现的低饱和度颜色，训练时还引入了类别重平衡（class rebalancing）机制，专门给稀有颜色加权，并通过"退火均值（annealed-mean）"将预测分布转换为最终颜色，在鲜艳度与空间一致性之间取得平衡。在百万级 ImageNet 图片上训练后，研究者设计了一套"图灵测试"来评估效果：让人类参与者区分真实彩色照片与算法生成结果，该方法成功骗过了 32% 的参与者，远超此前方法。更值得关注的是，这套自动上色框架还被证明是一种有效的自监督表示学习（self-supervised representation learning）方法，在多个特征学习基准上达到当时最优水平，为无标注视觉预训练提供了一条简洁可行的新路径。
原文 arXiv:1603.08511;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1603.08511v5