Hypercolumns for Object Segmentation and Fine-grained Localization
Bharath Hariharan University of California Berkeley Pablo Arbeláez Universidad de los Andes Colombia Ross Girshick Microsoft Research Redmond Jitendra Malik University of California Berkeley
Abstract
Recognition algorithms based on convolutional networks (CNNs) typically use the output of the last layer as a feature representation. However, the information in this layer may be too coarse spatially to allow precise localization. On the contrary, earlier layers may be precise in localization but will not capture semantics. To get the best of both worlds, we define the hypercolumn at a pixel as the vector of activations of all CNN units above that pixel. Using hypercolumns as pixel descriptors, we show results on three fine-grained localization tasks: simultaneous detection and segmentation [22], where we improve state-of-the-art from 49.7 mean APr [22] to 60.0, keypoint localization, where we get a 3.3 point boost over [20], and part labeling, where we show a 6.6 point gain over a strong baseline.
中文速览
深层卷积网络（CNN）的最后一层特征语义丰富却空间模糊，早期层定位精确却缺乏语义，两者各有短板。为此，研究者提出"超列"（hypercolumn）的概念：把输入图像某个像素位置正上方所有CNN层的激活值拼接成一个长向量，作为该像素的描述子，从而同时利用高层语义和低层空间细节。基于这一表示，他们将同步检测与分割、关键点定位、部件标注等精细定位任务统一转化为像素分类问题，并设计了一套插值分类器网格来捕捉位置相关的决策边界。实验结果显示，在同步检测与分割任务上，平均APr从49.7大幅提升至60.0，关键点定位和部件标注也分别获得3.3和6.6个百分点的增益。这项工作揭示了CNN多层特征的互补性，为后来众多密集预测（dense prediction）方法（如语义分割、实例分割）提供了重要的方法论基础。
原文 arXiv:1411.5752;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1411.5752v2