Poisoning and Backdooring Contrastive Learning
Nicholas Carlini Google、Andreas Terzis Google
Abstract
Multimodal contrastive learning methods like CLIP train on noisy and uncurated training datasets. This is cheaper than labeling datasets manually, and even improves out-of-distribution robustness. We show that this practice makes backdoor and poisoning attacks a significant threat. By poisoning just $0.01\%$ of a dataset (e.g., just 300 images of the 3 million-example Conceptual Captions dataset), we can cause the model to misclassify test images by overlaying a small patch. Targeted poisoning attacks, whereby the model misclassifies a particular test input with an adversarially-desired label, are even easier requiring control of $0.0001\%$ of the dataset (e.g., just three out of the 3 million images). Our attacks call into question whether training on noisy and uncurated Internet scrapes is desirable.
中文速览
像CLIP这样的多模态对比学习模型（multimodal contrastive learning）通常直接在从互联网爬取的海量噪声数据上训练，省去了人工标注的成本，但这也意味着任何人都可以悄悄在数据里动手脚。研究者发现，只需污染训练集中极少量的样本——对于拥有300万图文对的Conceptual Captions数据集，仅需注入300张图片（占比0.01%）就能让模型把任何带有特定触发图案的图片误分类为攻击者指定的类别（后门攻击，backdoor attack）；而要让模型针对某一特定测试图片产生错误预测（定向投毒攻击，targeted poisoning attack），甚至只需3张图片（占比0.0001%）。攻击之所以如此高效，是因为对比学习模型的训练集与测试集之间本身就存在较大的泛化差距，模型更容易过拟合那些精心构造的毒样本。这一发现对当前直接使用未经审查的互联网数据训练大规模视觉-语言模型的主流做法提出了严重的安全警示。
原文 arXiv:2106.09667;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2106.09667v2