Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
Nicolas Papernot and Patrick McDaniel Ian Goodfellow The Pennsylvania State University University Park, PA OpenAI San Francisco, CA
Abstract
Many machine learning models are vulnerable to adversarial examples: inputs that are specially crafted to cause a machine learning model to produce an incorrect output. Adversarial examples that affect one model often affect another model, even if the two models have different architectures or were trained on different training sets, so long as both models were trained to perform the same task. An attacker may therefore train their own substitute model, craft adversarial examples against the substitute, and transfer them to a victim model, with very little information about the victim. Recent work has further developed a technique that uses the victim model as an oracle to label a synthetic training set for the substitute, so the attacker need not even collect a training set to mount the attack. We extend these recent techniques using reservoir sampling to greatly enhance the efficiency of the training procedure for the substitute model. We introduce new transferability attacks between previously unexplored (substitute, victim) pairs of machine learning model classes, most notably SVMs and decision trees. We demonstrate our attacks on two commercial machine learning classification
中文速览
对抗样本（adversarial examples）存在跨模型迁移的特性，攻击者可以在不了解目标模型内部结构的情况下，训练一个替代模型（substitute model）来生成对抗样本，再将其迁移攻击真实目标——这篇工作在此基础上，将黑盒攻击的范围从深度神经网络大幅扩展到支持向量机、决策树、逻辑回归、近邻算法等几乎所有主流机器学习模型，并首次为不可微模型设计了对抗样本生成方法。研究者引入了蓄水池采样（reservoir sampling）来大幅提升替代模型的训练效率，将所需的目标模型查询次数压缩至仅 800 次。实验结果表明，以逻辑回归替代模型对亚马逊和谷歌的商业分类服务发动黑盒攻击，误分类率分别高达 96.19% 和 88.94%。这项工作说明，无论模型架构如何，现有机器学习系统普遍面临系统性黑盒攻击的威胁，迫切需要在生产环境中对模型输入进行严格验证。
原文 arXiv:1605.07277;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1605.07277v1