Make Up Your Mind! Adversarial Generation of Inconsistent Natural Language Explanations
Oana-Maria Camburu1,4 Brendan Shillingford1,2 Pasquale Minervini3 Thomas Lukasiewicz1,4 Phil Blunsom1,2 1 University of Oxford 2 DeepMind, London 3 University College London 4 Alan Turing Institute, London
Abstract
To increase trust in artificial intelligence systems, a promising research direction consists of designing neural models capable of generating natural language explanations for their predictions. In this work, we show that such models are nonetheless prone to generating mutually inconsistent explanations, such as “Because there is a dog in the image.” and “Because there is no dog in the [same] image.”, exposing flaws in either the decision-making process of the model or in the generation of the explanations. We introduce a simple yet effective adversarial framework for sanity checking models against the generation of inconsistent natural language explanations. Moreover, as part of the framework, we address the problem of adversarial attacks with full target sequences, a scenario that was not previously addressed in sequence-to-sequence attacks. Finally, we apply our framework on a state-of-the-art neural natural language inference model that provides natural language explanations for its predictions. Our framework shows that this model is capable of generating a significant number of inconsistent explanations.
中文速览
让AI系统说清楚"为什么这么判断"本是提升可信度的好思路，但这类能生成自然语言解释（natural language explanation）的神经模型存在一个隐患：它们可能对同一张图或同一段文本同时给出"图中有狗"和"图中没有狗"这样自相矛盾的解释。为了系统地暴露这一缺陷，研究者提出了一套对抗性检测框架（adversarial framework）：先用逻辑规则（如否定、模板替换）构造与原解释矛盾的目标解释，再训练一个"反向解释模型"（RevExpl）从目标解释反推出新输入，最后检验模型是否真的为这个新输入生成了矛盾的解释——这也是首次在序列到序列攻击中将完整目标序列作为攻击目标的尝试。将该框架应用于当时最先进的自然语言推理解释模型（EtPA）后，在测试集中发现了逾千对自相矛盾的解释，有力地证明这类模型的可靠性远不如表面看起来那么好，对AI可信度研究具有重要的警示意义。
原文 arXiv:1910.03065;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1910.03065v3