Poisoning Web-Scale Training Datasets is Practical
Nicholas Carlini1 Matthew Jagielski1 Christopher A. Choquette-Choo1 Daniel Paleka2 Will Pearce3 Hyrum Anderson4 Andreas Terzis1 Kurt Thomas5 Florian Tramèr2 1Google DeepMind 2ETH Zurich 3NVIDIA 4Robust Intelligence 5Google
Abstract
Deep learning models are often trained on distributed, web-scale datasets crawled from the internet. In this paper, we introduce two new dataset poisoning attacks that intentionally introduce malicious examples to a model’s performance. Our attacks are immediately practical and could, today, poison $10{}$ popular datasets. Our first attack, split-view poisoning, exploits the mutable nature of internet content to ensure a dataset annotator’s initial view of the dataset differs from the view downloaded by subsequent clients. By exploiting specific invalid trust assumptions, we show how we could have poisoned $0.01\%$ of the LAION-400M or COYO-700M datasets for just $60 USD. Our second attack, frontrunning poisoning, targets web-scale datasets that periodically snapshot crowd-sourced content—such as Wikipedia—where an attacker only needs a time-limited window to inject malicious examples. In light of both attacks, we notify the maintainers of each affected dataset and recommended several low-overhead defenses.
中文速览
大规模网络爬取的图像-文本数据集（如LAION-400M）正被广泛用于训练最先进的深度学习模型，但这类数据集本身存在严重安全隐患。研究者提出了两种全新的数据集投毒攻击：一是"分裂视图投毒"——利用互联网域名会过期并被他人购买的特性，在数据集维护者建立索引之后、用户下载之前，悄悄替换URL所指向的内容，使用户实际下载到与维护者当初看到的完全不同的恶意数据；二是"抢跑投毒"——针对Wikipedia等定期生成快照的众包内容数据集，攻击者只需在快照生成前的短暂窗口内注入恶意内容，即使事后被版主撤销，恶意样本也已永久保存在快照里。实验表明，仅花费60美元，攻击者就能在2023年对LAION-400M或COYO-700M中0.01%的样本完成投毒，且这一比例已足够影响模型行为。这项研究首次将数据投毒从"理论可行"推进到"实际可操作"，敲响了当前网络规模数据集缺乏完整性保护的警钟，研究者也因此通知了10个受影响数据集的维护方并推动了多项防御措施的落地。
原文 arXiv:2302.10149;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2302.10149v2