Class-Imbalanced Semi-Supervised Learning
Minsung Hyun Jisoo Jeong Nojun Kwak
Abstract
Semi-Supervised Learning (SSL) has achieved great success in overcoming the difficulties of labeling and making full use of unlabeled data. However, SSL has a limited assumption that the numbers of samples in different classes are balanced, and many SSL algorithms show lower performance for the datasets with the imbalanced class distribution. In this paper, we introduce a task of class-imbalanced semi-supervised learning (CISSL), which refers to semi-supervised learning with class-imbalanced data. In doing so, we consider class imbalance in both labeled and unlabeled sets. First, we analyze existing SSL methods in imbalanced environments and examine how the class imbalance affects SSL methods. Then we propose Suppressed Consistency Loss (SCL), a regularization method robust to class imbalance. Our method shows better performance than the conventional methods in the CISSL environment. In particular, the more severe the class imbalance and the smaller the size of the labeled data, the better our method performs.
中文速览
标注数据不足又存在类别不平衡，是现实场景中机器学习面临的双重难题，而以往的半监督学习（Semi-Supervised Learning, SSL）方法几乎默认各类别样本数量均衡，一旦遇到长尾分布的数据就会明显掉点。本文正式定义了"类别不平衡半监督学习"（Class-Imbalanced Semi-Supervised Learning, CISSL）这一任务，并通过玩具实验和数学分析揭示了现有SSL方法失效的根本原因：一致性正则化（consistency regularization）会把决策边界过度推向少数类的高密度区域，造成少数类大量误分类。基于这一发现，作者提出了"抑制一致性损失"（Suppressed Consistency Loss, SCL）方法，通过限制决策边界在少数类区域的过度平滑来缓解类别不平衡的负面影响。在多个基准数据集以及目标检测任务上的实验表明，SCL相比现有SSL和类别不平衡学习方法均取得更优的性能，且类别不平衡越严重、有标注数据越少时，优势越显著。这项工作填补了半监督学习与不平衡学习之间的空白，为实际应用中标注稀缺且数据分布不均的场景提供了更可靠的解决方案。
原文 arXiv:2002.06815;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2002.06815v1