Quantifying Memorization Across Neural Language Models
Nicholas Carlini Authors ordered alphabetically. Google Research Daphne Ippolito Google Research University of Pennsylvania Matthew Jagielski Google Research Katherine Lee Google Research Cornell University Florian Tramèr Google Research Chiyuan Zhang Google Research
Abstract
Large language models (LMs) have been shown to memorize parts of their training data, and when prompted appropriately, they will emit the memorized training data verbatim. This is undesirable because memorization violates privacy (exposing user data), degrades utility (repeated easy-to-memorize text is often low quality), and hurts fairness (some texts are memorized over others).
中文速览
大型语言模型（LM）会记住训练数据中的内容，攻击者只需给出合适的提示就能让模型逐字复现这些数据，但此前研究对"记忆了多少"的估计极为粗糙。这项工作系统地测量了记忆程度与三个因素之间的对数线性关系：模型参数规模越大、同一样本在训练集中重复次数越多、用于提示的上下文越长，模型"吐出"训练数据的概率就越高，三者均呈显著的对数线性增长。实验在 GPT-Neo 等多个模型家族上进行，结果显示 60亿参数的 GPT-J 至少记住了训练集 The Pile 中 1% 的内容，远高于此前研究估计的极小比例。这一发现说明随着模型规模持续扩大，训练数据的隐私泄露风险将比人们预想的更为严峻，需要在模型设计和训练流程中主动采取去重、差分隐私等缓解措施。
原文 arXiv:2202.07646;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2202.07646v3