Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning
Battista Biggio Fabio Roli Department of Electrical and Electronic Engineering, University of Cagliari, Italy Pluribus One, Cagliari, Italy
Abstract
Learning-based pattern classifiers, including deep networks, have shown impressive performance in several application domains, ranging from computer vision to cybersecurity. However, it has also been shown that adversarial input perturbations carefully crafted either at training or at test time can easily subvert their predictions. The vulnerability of machine learning to such wild patterns (also referred to as adversarial examples), along with the design of suitable countermeasures, have been investigated in the research field of adversarial machine learning. In this work, we provide a thorough overview of the evolution of this research area over the last ten years and beyond, starting from pioneering, earlier work on the security of non-deep learning algorithms up to more recent work aimed to understand the security properties of deep learning algorithms, in the context of computer vision and cybersecurity tasks. We report interesting connections between these apparently-different lines of work, highlighting common misconceptions related to the security evaluation of machine-learning algorithms. We review the main threat models and attacks defined to this end, and discuss the mai
中文速览
机器学习分类器在图像识别、恶意软件检测等领域表现出色，但攻击者只需对输入数据进行精心设计的微小扰动，就能让这些系统做出错误判断——这类样本被称为对抗样本（adversarial examples）。这篇综述系统梳理了对抗机器学习（adversarial machine learning）领域近十余年的发展脉络，从2004年前后针对垃圾邮件过滤器的早期攻击研究，一直追溯到近年来针对深度神经网络的各类攻击与防御工作，重点揭示了两条研究线索之间被长期忽视的内在联系，并纠正了"对抗样本研究始于2014年"这一普遍误解。文章围绕"了解攻击者、主动评估、设计防御"三条主线，系统介绍了威胁建模框架、训练阶段的数据投毒（poisoning）攻击与测试阶段的规避（evasion）攻击，以及现有防御手段的局限性。这项工作的价值在于为研究者提供了一套统一的安全评估视角，有助于推动更健壮的机器学习系统的设计，对计算机视觉和网络安全两个应用领域都具有重要参考意义。
原文 arXiv:1712.03141;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1712.03141v2