An Empirical Study of the Collapsing Problem in Semi-Supervised 22D Human Pose Estimation

中文速览

半监督学习用于2D人体姿态估计时，常因前景关节点像素远少于背景而发生“坍塌”，把未标注图像几乎所有像素都预测成背景。为此，方法为每张未标注图像构造一对难度不同的增强，用较容易增强得到的更可靠姿态热图作为教师信号，只指导较难增强的预测，并停止教师分支的梯度传播。实验表明，该策略能稳定训练、避免坍塌，在少量标注数据下将平均精度从31.5%提升到44.6%，并在COCO、MPII和H36M等数据集及现有强姿态模型上持续带来收益。它说明一致性半监督学习不能忽视类别极度不均衡的问题，也提供了一种简单、通用的办法来利用海量未标注人体图像提升模型泛化能力。

摘要

Most semi-supervised learning models are consistency-based, which leverage unlabeled images by maximizing the similarity between different augmentations of an image. But when we apply them to human pose estimation that has extremely imbalanced class distribution, they often collapse and predict every pixel in unlabeled images as background. We find this is because the decision boundary passes the high-density areas of the minor class so more and more pixels are gradually mis-classified as background. In this work, we present a surprisingly simple approach to drive the model to learn in the correct direction. For each image, it composes a pair of easy-hard augmentations and uses the more accurate predictions on the easy image to teach the network to learn pose information of the hard one. The accuracy superiority of teaching signals allows the network to be “monotonically” improved which effectively avoids collapsing. We apply our method to the state-of-the-art pose estimators and it further improves their performance on three public datasets. The source code and pretrained models have been released at https://github.com/xierc/Semi_Human_Pose.

术语表

semi-supervised learning (SSL)

半监督学习（SSL）

consistency-based learning

基于一致性的学习

human pose estimation

人体姿态估计

2D human pose estimation

二维人体姿态估计

3D pose modeling

三维姿态建模

heatmap-based framework

基于热图的框架

pseudo labeling

伪标签

pseudo labels

伪标签

data augmentation

数据增强

easy-hard augmentation pair

易-难增强对

collapsing problem

坍塌问题

class imbalance

类别不平衡

decision boundary

决策边界

minor class

少数类

teaching signal

教学信号

consistency regularization

一致性正则化

stop-gradient

停止梯度

mean-teacher model

均值教师模型

Pi model

Pi模型

Exponential Moving Average (EMA)

指数移动平均（EMA）

BYOL

BYOL

SimSiam

SimSiam

mean Average Precision (AP)

平均精度（AP）

COCO

COCO数据集

MPII

MPII数据集

H36M

H36M数据集

semi-supervised pre-training

半监督预训练

domain adaptation

领域自适应