Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU Models
Mengnan Du1, Varun Manjunatha2, Rajiv Jain2, Ruchi Deshpande3, Franck Dernoncourt2, Jiuxiang Gu2, Tong Sun2 and Xia Hu1 1Texas A、M University 2Adobe Research 3Adobe Document Cloud Most of the work was done while the first author was an intern at Adobe Research.
Abstract
Recent studies indicate that NLU models are prone to rely on shortcut features for prediction, without achieving true language understanding. As a result, these models fail to generalize to real-world out-of-distribution data. In this work, we show that the words in the NLU training set can be modeled as a long-tailed distribution. There are two findings: 1) NLU models have strong preference for features located at the head of the long-tailed distribution, and 2) Shortcut features are picked up during very early few iterations of the model training. These two observations are further employed to formulate a measurement which can quantify the shortcut degree of each training sample. Based on this shortcut measurement, we propose a shortcut mitigation framework LTGR, to suppress the model from making overconfident predictions for samples with large shortcut degree. Experimental results on three NLU benchmarks demonstrate that our long-tailed distribution explanation accurately reflects the shortcut learning behavior of NLU models. Experimental analysis further indicates that LTGR can improve the generalization accuracy on OOD data, while preserving the accuracy on in-distribution dat
中文速览
自然语言理解（NLU）模型（如 BERT）虽然在标准测试集上表现亮眼，却常常依赖数据集中的"捷径特征"（shortcut features）——比如某些词与标签的虚假统计关联——而非真正理解语义，导致在真实分布外（out-of-distribution, OOD）数据上表现大幅下滑。作者发现训练集中词语的统计特征呈长尾分布（long-tailed distribution），头部是高频但信息量低的功能词，尾部才是真正有价值的语义特征；NLU 模型天然偏爱头部捷径特征，且这些特征在训练极早期就被模型"学走了"。基于这两个发现，作者设计了一套量化每个训练样本"捷径程度"的指标，并提出名为 LTGR 的缓解框架：通过自知识蒸馏（self knowledge distillation），对捷径程度高的样本压低模型的过度自信预测，迫使模型转而关注真正相关的语义特征。在 FEVER、MNLI、QA-NLI 等多个基准的 OOD 测试集上，LTGR 均显著提升了泛化准确率，同时不牺牲原始分布内的性能，为解决 NLU 模型的捷径依赖问题提供了可解释且有效的新思路。
原文 arXiv:2103.06922;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2103.06922v3