Adversarial examples in the physical world
Alexey Kurakin Google Brain、Ian J. Goodfellow OpenAI、Samy Bengio Google Brain
Abstract
Most existing machine learning classifiers are highly vulnerable to adversarial examples. An adversarial example is a sample of input data which has been modified very slightly in a way that is intended to cause a machine learning classifier to misclassify it. In many cases, these modifications can be so subtle that a human observer does not even notice the modification at all, yet the classifier still makes a mistake. Adversarial examples pose security concerns because they could be used to perform an attack on machine learning systems, even if the adversary has no access to the underlying model. Up to now, all previous work has assumed a threat model in which the adversary can feed data directly into the machine learning classifier. This is not always the case for systems operating in the physical world, for example those which are using signals from cameras and other sensors as input. This paper shows that even in such physical world scenarios, machine learning systems are vulnerable to adversarial examples. We demonstrate this by feeding adversarial images obtained from a cell-phone camera to an ImageNet Inception classifier and measuring the classification accuracy of the syst
中文速览
深度神经网络很容易被"对抗样本"（adversarial examples）欺骗——只需对图片做几乎肉眼不可见的微小改动，分类器就会出错——但此前所有研究都假设攻击者能把数据直接输入模型，没人知道这种威胁在现实物理场景中是否依然成立。这篇论文用手机摄像头拍下打印或显示在屏幕上的对抗样本，再送入 ImageNet Inception v3 分类器，系统性地测量了经过"相机这一关"之后对抗样本的攻击存活率。结果表明，大量对抗样本即便经过拍照、打印等物理过程的干扰，仍然能让分类器持续出错，而且研究者无需对生成方法做任何专门改造。这一发现证明机器学习系统在摄像头、传感器驱动的真实世界应用（如机器人、监控、手机 App）中同样面临对抗攻击威胁，为后续设计更鲁棒的物理世界 AI 安全防御提供了重要依据。
原文 arXiv:1607.02533;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1607.02533v4