Training deep neural networks on noisy labels with bootstrapping
Scott E. Reed、Honglak Lee Dept. of Electrical Engineering and Computer Science, University of Michigan Ann Arbor, MI, USA、Dragomir Anguelov, Christian Szegedy, Dumitru Erhan、Andrew Rabinovich Google, Inc. Mountain View, CA, USA
Abstract
Current state-of-the-art deep learning systems for visual object recognition and detection use purely supervised training with regularization such as dropout to avoid overfitting. The performance depends critically on the amount of labeled examples, and in current practice the labels are assumed to be unambiguous and accurate. However, this assumption often does not hold; e.g. in recognition, class labels may be missing; in detection, objects in the image may not be localized; and in general, the labeling may be subjective. In this work we propose a generic way to handle noisy and incomplete labeling by augmenting the prediction objective with a notion of consistency. We consider a prediction consistent if the same prediction is made given similar percepts, where the notion of similarity is between deep network features computed from the input data. In experiments we demonstrate that our approach yields substantial robustness to label noise on several datasets. On MNIST handwritten digits, we show that our model is robust to label corruption. On the Toronto Face Database, we show that our model handles well the case of subjective labels in emotion recognition, achieving state-of-th
中文速览
深度学习做图像识别时，训练标签往往存在噪声、缺失或主观性（比如不同标注者对同一张人脸的情绪判断不同），这严重影响模型质量。这篇论文提出了一种通用方法：在原有的预测损失之外，加入一个"感知一致性（perceptual consistency）"目标，核心思想是——如果网络特征空间中两张图片看起来很像，模型就应该给出相同的预测；当训练标签与这一判断矛盾时，模型可以逐渐"纠正"自己看到的标签，即用模型当前预测与原始标签的加权组合来替换训练目标，也就是所谓的自举（bootstrapping）。作者在三个任务上验证了这一方法：MNIST手写数字（人工加入标签噪声时依然鲁棒）、多伦多人脸数据库情绪识别（达到当时最优，且无需修改方法即可利用无标签数据）、以及ILSVRC2014目标检测（在极深网络和高分辨率图像上均取得性能提升）。这项工作的价值在于，它几乎不需要改动现有深度网络结构，就能以最小的工程代价处理大规模训练数据中普遍存在的标签不完善问题，为弱监督深度学习提供了一条实用路径。
原文 arXiv:1412.6596;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1412.6596v3