An Empirical Study of the Collapsing Problem in Semi-Supervised 222D Human Pose Estimation
Rongchang Xie1, Chunyu Wang2, Wenjun Zeng2, Yizhou Wang3 1Center for Data Science, Peking University 2Microsoft Research Asia 3 Center on Frontiers of Computing Studies, CS Dept., Peking University {rongchangxie, {chnuwa,
Abstract
Most semi-supervised learning models are consistency-based, which leverage unlabeled images by maximizing the similarity between different augmentations of an image. But when we apply them to human pose estimation that has extremely imbalanced class distribution, they often collapse and predict every pixel in unlabeled images as background. We find this is because the decision boundary passes the high-density areas of the minor class so more and more pixels are gradually mis-classified as background. In this work, we present a surprisingly simple approach to drive the model to learn in the correct direction. For each image, it composes a pair of easy-hard augmentations and uses the more accurate predictions on the easy image to teach the network to learn pose information of the hard one. The accuracy superiority of teaching signals allows the network to be “monotonically” improved which effectively avoids collapsing. We apply our method to the state-of-the-art pose estimators and it further improves their performance on three public datasets. The source code and pretrained models have been released at https://github.com/xierc/Semi_Human_Pose.
中文速览
人体姿态估计（human pose estimation）任务中存在严重的类别不平衡问题，直接把主流半监督学习方法套用到该任务上，模型会迅速崩溃、把所有像素都预测成背景，反而不如只用有标注数据训练的效果。本文发现崩溃根源在于：一致性约束在类别严重不平衡时会让决策边界偏移进关节点这类稀少类别的高密度区域，导致越来越多像素被误判为背景。为此，作者提出了一个出人意料简单的解决方案：对每张无标注图像构造一对"易-难"增强（easy-hard augmentation），用网络在易增强图上更准确的预测去监督它在难增强图上的学习，同时切断反向梯度从难到易方向的传播，从而让模型始终朝更准确的方向单调改进、不再崩溃。实验在COCO、MPII和H36M三个公开数据集上验证了该方法，仅用1K标注图像时平均精度（AP）从31.5%大幅提升至44.6%，远超伪标签类基线，且在使用全量标注数据时依然能借助无标注图像进一步涨点，对推动半监督人体姿态估计研究具有重要的实践价值。
原文 arXiv:2011.12498;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2011.12498v4