Quantifying Memorization Across Neural Language Models
Nicholas Carlini Thanks: Authors ordered alphabetically. Affiliation: Google Research Daphne Ippolito Affiliation: Google Research Affiliation: University of Pennsylvania Matthew Jagielski Affiliation: Google Research Katherine Lee Affiliation: Google Research Affiliation: Cornell University Florian Tramèr Affiliation: Google Research Chiyuan Zhang Affiliation: Google Research
Abstract
Large language models (LMs) have been shown to memorize parts of their training data, and when prompted appropriately, they will emit the memorized training data verbatim. This is undesirable because memorization violates privacy (exposing user data), degrades utility (repeated easy-to-memorize text is often low quality), and hurts fairness (some texts are memorized over others).
原文 arXiv:2202.07646;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2202.07646v3