Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients
Andrew Slavin Ross Finale Doshi-Velez Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, MA 02138, USA
Abstract
Deep neural networks have proven remarkably effective at solving many classification problems, but have been criticized recently for two major weaknesses: the reasons behind their predictions are uninterpretable, and the predictions themselves can often be fooled by small adversarial perturbations. These problems pose major obstacles for the adoption of neural networks in domains that require security or transparency. In this work, we evaluate the effectiveness of defenses that differentiably penalize the degree to which small changes in inputs can alter model predictions. Across multiple attacks, architectures, defenses, and datasets, we find that neural networks trained with this input gradient regularization exhibit robustness to transferred adversarial examples generated to fool all of the other models. We also find that adversarial examples generated to fool gradient-regularized models fool all other models equally well, and actually lead to more “legitimate,” interpretable misclassifications as rated by people (which we confirm in a human subject experiment). Finally, we demonstrate that regularizing input gradients makes them more naturally interpretable as rationales for mo
中文速览
训练好的深度神经网络存在两大痛点：预测结果难以解释，以及容易被微小的对抗扰动（adversarial perturbation）欺骗。研究者提出用"输入梯度正则化"（input gradient regularization）来同时解决这两个问题——在训练时直接惩罚模型输出对输入变化的敏感程度，让模型的局部线性响应保持平滑。实验在MNIST、SVHN、notMNIST三个数据集和多种网络结构上与蒸馏防御、对抗训练等主流方法对比，发现梯度正则化模型对"迁移攻击"（针对其他模型生成的对抗样本）表现出显著更强的鲁棒性，而针对梯度正则化模型生成的对抗样本也会同等有效地迁移欺骗其他模型，并且这些样本在人类眼中看起来更像目标类别——人类实验证实了这一点。更重要的是，经过梯度正则化的模型，其输入梯度本身就变得清晰可读，可直接作为预测的可视化解释，无需额外的平滑后处理，从而揭示了模型可解释性与对抗鲁棒性之间存在内在关联。
原文 arXiv:1711.09404;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1711.09404v1