Explain Yourself! Leveraging Language Models for Commonsense Reasoning
Nazneen Fatema Rajani Bryan McCann Caiming Xiong Richard Socher Salesforce Research Palo Alto, CA, 94301
Abstract
Deep learning models perform poorly on tasks that require commonsense reasoning, which often necessitates some form of world-knowledge or reasoning over information not immediately present in the input. We collect human explanations for commonsense reasoning in the form of natural language sequences and highlighted annotations in a new dataset called Common Sense Explanations (CoS-E). We use CoS-E to train language models to automatically generate explanations that can be used during training and inference in a novel Commonsense Auto-Generated Explanation (CAGE) framework. CAGE improves the state-of-the-art by $10\%$ on the challenging CommonsenseQA task. We further study commonsense reasoning in DNNs using both human and auto-generated explanations including transfer to out-of-domain tasks. Empirical results indicate that we can effectively leverage language models for commonsense reasoning.
中文速览
大型深度学习模型在常识推理（commonsense reasoning）任务上长期表现欠佳，根本原因在于缺乏对世界知识的有效利用。为此，研究者通过众包平台收集了一批人工撰写的自然语言解释，构建了 CoS-E（Common Sense Explanations）数据集，并以此为基础提出了 CAGE（Commonsense Auto-Generated Explanation）框架：先用 GPT 语言模型学习生成常识解释，再将这些自动生成的解释拼接到问题输入中，最终交由 BERT 模型做多选题判断。在基准测试 CommonsenseQA 上，该方法将准确率提升了约 10 个百分点，达到当时最优水平，并在迁移至其他领域任务时同样表现出色。这项工作的意义在于：它证明了让语言模型先"说出推理过程"再作答，是一种切实可行的常识推理增强路径，同时 CoS-E 数据集也为后续研究模型是否真正理解常识提供了可解释性分析工具。
原文 arXiv:1906.02361;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1906.02361v1