Can Rationalization Improve Robustness?
Howard Chen Jacqueline He Karthik Narasimhan Danqi Chen Department of Computer Science, Princeton University {howardchen, karthikn,
Abstract
A growing line of work has investigated the development of neural NLP models that can produce rationales—subsets of input that can explain their model predictions. In this paper, we ask whether such rationale models can provide robustness to adversarial attacks in addition to their interpretable nature. Since these models need to first generate rationales (“rationalizer”) before making predictions (“predictor”), they have the potential to ignore noise or adversarially added text by simply masking it out of the generated rationale. To this end, we systematically generate various types of ‘AddText’ attacks for both token and sentence-level rationalization tasks and perform an extensive empirical evaluation of state-of-the-art rationale models across five different tasks. Our experiments reveal that rationale models show promise in improving robustness but struggle in certain scenarios—e.g., when the rationalizer is sensitive to position bias or lexical choices of the attack text. Further, leveraging human rationales as supervision does not always translate to better performance. Our study is a first step towards exploring the interplay between interpretability and robustness in the r
中文速览
理性化模型（rationale model）在做预测前会先从输入中挑出关键片段作为"理由"，研究者想知道这种先筛选、再预测的机制能不能同时带来对抗攻击下的鲁棒性。为此，他们针对五项NLP任务系统设计了多种"AddText"攻击——即在测试时向输入中插入随机文本、维基百科句子或对抗性构造句子——并对多个最先进的理性化模型展开全面评测。结果显示，理性化模型确实有一定的抗攻击潜力，在最强攻击下任务性能下降通常不超过10%，而普通全文本模型则下降11%–27%；但这种优势并不稳定，模型对攻击文本的插入位置和词汇选择都相当敏感，引入人工标注的理由作为监督也未必能提升鲁棒性。这项工作首次系统揭示了"可解释性"与"鲁棒性"在理性化框架下的内在关联，为设计更可靠的可解释NLP模型提供了重要参考。
原文 arXiv:2204.11790;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2204.11790v2