A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
Kimin Lee1, Kibok Lee2, Honglak Lee3,2, Jinwoo Shin1,4 1Korea Advanced Institute of Science and Technology (KAIST) 2University of Michigan 3Google Brain 4AItrics
Abstract
Detecting test samples drawn sufficiently far away from the training distribution statistically or adversarially is a fundamental requirement for deploying a good classifier in many real-world machine learning applications. However, deep neural networks with the softmax classifier are known to produce highly overconfident posterior distributions even for such abnormal samples. In this paper, we propose a simple yet effective method for detecting any abnormal samples, which is applicable to any pre-trained softmax neural classifier. We obtain the class conditional Gaussian distributions with respect to (low- and upper-level) features of the deep models under Gaussian discriminant analysis, which result in a confidence score based on the Mahalanobis distance. While most prior methods have been evaluated for detecting either out-of-distribution or adversarial samples, but not both, the proposed method achieves the state-of-the-art performances for both cases in our experiments. Moreover, we found that our proposed method is more robust in harsh cases, e.g., when the training dataset has noisy labels or small number of samples. Finally, we show that the proposed method enjoys broader u
中文速览
深度神经网络在面对训练分布之外的样本（分布外样本，OOD）或对抗样本时，其softmax分类器往往输出虚假的高置信度，导致安全隐患。针对这一问题，研究者提出了一种基于马哈拉诺比斯距离（Mahalanobis distance）的置信度评分方法：在高斯判别分析框架下，用预训练模型各层特征拟合类条件高斯分布，再以测试样本到最近类分布的马哈拉诺比斯距离作为异常检测得分，并结合输入扰动预处理和多层特征融合进一步提升检测效果。实验在DenseNet、ResNet等主流网络和CIFAR、ImageNet等多个数据集上验证，该方法在OOD检测和对抗样本检测两项任务上均超越当时最优方法，例如在ResNet上检测CIFAR-100分布外样本的真负率从45.6%提升至90.9%，检测CW对抗攻击的真负率从82.9%提升至95.8%。这项工作的重要意义在于，它提供了一个无需重训练、适用于任意预训练softmax分类器的通用异常检测框架，还可直接扩展到类增量学习等场景，为提升深度学习系统在真实环境中的可靠性与安全性提供了切实可行的解决方案。
原文 arXiv:1807.03888;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1807.03888v2