Inoculation by Fine-Tuning: A Method for Analyzing Challenge Datasets
Nelson F. Liu♠♡♣ Roy Schwartz♠♣ Noah A. Smith♠♣ ♠Paul G. Allen School of Computer Science、Engineering, University of Washington, Seattle, WA, USA ♡Department of Linguistics, University of Washington, Seattle, WA, USA ♣Allen Institute for Artificial Intelligence, Seattle, WA, USA
Abstract
Several datasets have recently been constructed to expose brittleness in models trained on existing benchmarks. While model performance on these challenge datasets is significantly lower compared to the original benchmark, it is unclear what particular weaknesses they reveal. For example, a challenge dataset may be difficult because it targets phenomena that current models cannot capture, or because it simply exploits blind spots in a model’s specific training set. We introduce inoculation by fine-tuning, a new analysis method for studying challenge datasets by exposing models (the metaphorical patient) to a small amount of data from the challenge dataset (a metaphorical pathogen) and assessing how well they can adapt. We apply our method to analyze the NLI “stress tests” Naik et al. (2018) and the Adversarial SQuAD dataset Jia and Liang (2017). We show that after slight exposure, some of these datasets are no longer challenging, while others remain difficult. Our results indicate that failures on challenge datasets may lead to very different conclusions about models, training datasets, and the challenge datasets themselves.
中文速览
当前NLP模型在各种"挑战数据集"上表现大幅下滑，但我们并不清楚这究竟是因为模型本身存在根本性缺陷，还是仅仅因为原始训练数据覆盖不足。作者提出了一种名为"微调接种"（inoculation by fine-tuning）的分析方法：把模型比作病人，让它接触少量挑战数据集样本（疫苗），再观察它能否恢复正常表现，以此判断问题出在训练数据还是模型本身。将该方法应用于NLI压力测试和Adversarial SQuAD后发现，词汇重叠、否定等挑战集只需百余个样本便能让模型性能完全恢复，说明问题源于训练数据的盲点；而拼写错误和长度不匹配挑战则无论如何微调都难以突破，暴露的是模型的根本性弱点；数值推理和Adversarial SQuAD虽可恢复，但代价是原始任务性能下滑，进一步揭示这些挑战集本身存在标注捷径。这项工作提供了一把简单易用的"诊断工具"，帮助研究者区分究竟是数据不够多样还是模型能力不足，从而更有针对性地推动NLP进步。
原文 arXiv:1904.02668;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1904.02668v4