What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation
Vitaly Feldman Apple Equal contribution.Part of this work done while the author was at Google Research, Brain Team. Chiyuan Zhang Google Research, Brain Team
Abstract
Deep learning algorithms are well-known to have a propensity for fitting the training data very well and often fit even outliers and mislabeled data points. Such fitting requires memorization of training data labels, a phenomenon that has attracted significant research interest but has not been given a compelling explanation so far. A recent work of [Fel19] proposes a theoretical explanation for this phenomenon based on a combination of two insights. First, natural image and data distributions are (informally) known to be long-tailed, that is have a significant fraction of rare and atypical examples. Second, in a simple theoretical model such memorization is necessary for achieving close-to-optimal generalization error when the data distribution is long-tailed. However, no direct empirical evidence for this explanation or even an approach for obtaining such evidence were given.
中文速览
深度神经网络在训练时会死记硬背（memorize）部分样本的标签，传统理论认为这是过拟合的表现，与泛化能力相悖，但现实中这种记忆现象普遍存在且并不总是有害——为什么？一种叫做"长尾理论（long tail theory）"的假说认为，真实数据分布存在大量稀有、非典型样本，模型必须记住这些样本的标签才能在它们所代表的罕见子群体上取得好的泛化效果。本文设计了一套实验框架，通过高效的子采样估计器来量化每个训练样本的"记忆值"以及每对训练-测试样本之间的"影响值"，在 CIFAR-100 和 ImageNet 等标准数据集上系统检验了上述理论。实验结果表明，被高度记忆的训练样本对测试准确率的边际贡献显著高于随机样本，并且存在大量"一对一"的高影响训练-测试样本对——单个训练样本显著提升特定测试样本的预测准确率，这些样本对在视觉上往往呈现出清晰可辨的相似性。这不仅为长尾理论提供了迄今最直接的定量与直觉证据，也提示我们盲目抑制记忆现象可能会损害模型在稀有类别上的泛化能力。
原文 arXiv:2008.03703;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2008.03703v1