Expansion and Shrinkage of Localization for Weakly-Supervised Semantic Segmentation
Jinlong Li1,2,∗ Zequn Jie2,∗ Xu Wang1 Xiaolin Wei2 Lin Ma2 1 College of Computer Science and Software Engineering, Shenzhen University 2 Meituan ∗These authors contributed equally
Abstract
Generating precise class-aware pseudo ground-truths, a.k.a, class activation maps (CAMs), is essential for weakly-supervised semantic segmentation. The original CAM method usually produces incomplete and inaccurate localization maps. To tackle with this issue, this paper proposes an Expansion and Shrinkage scheme based on the offset learning in the deformable convolution, to sequentially improve the recall and precision of the located object in the two respective stages. In the Expansion stage, an offset learning branch in a deformable convolution layer, referred as “expansion sampler” seeks for sampling increasingly less discriminative object regions，driven by an inverse supervision signal that maximizes image-level classification loss. The located more complete object in the Expansion stage is then gradually narrowed down to the final object region during the Shrinkage stage. In the Shrinkage stage, the offset learning branch of another deformable convolution layer, referred as “shrinkage sampler”, is introduced to exclude the false positive background regions attended in the Expansion stage to improve the precision of the localization maps. We conduct various experiments on PASC
中文速览
弱监督语义分割（weakly-supervised semantic segmentation）领域长期面临一个核心难题：仅凭图像级别的类别标签生成的类激活图（class activation map, CAM）只能覆盖目标物体最显著的局部区域，既不完整又不精准，导致后续分割模型的伪标签质量低下。为此，这篇论文提出了一套名为"扩张-收缩偏移学习"（Expansion and Shrinkage with Offset Learning, ESOL）的两阶段训练框架：扩张阶段在冻结骨干网络的前提下，利用可变形卷积（deformable convolution）的偏移学习分支以"最大化分类损失"这一反向信号驱动网络主动采样那些原本被忽略的低判别性区域，从而尽可能覆盖完整物体；收缩阶段则再引入一个可变形卷积层，以正常分类信号将高召回率的区域精确收窄至真正的物体区域，同时用特征截断（feature clipping）策略消除扩张阶段遗留的激活偏差，避免网络再次退化回初始的显著区域。在 PASCAL VOC 2012 和 MS COCO 2014 两个标准基准上的实验表明，该方法生成的伪标签质量显著优于现有方法，最终分割性能达到当时的最优水平，为如何用极低成本的标注信号训练出高质量分割模型提供了一条清晰可行的新路径。
原文 arXiv:2209.07761;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2209.07761v2