Fusing Context Into Knowledge Graph for Commonsense Question Answering
Yichong Xu , Chenguang Zhu∗, Ruochen Xu, Yang Liu, Michael Zeng, Xuedong Huang Microsoft Cognitive Services Research Group Equal contribution
Abstract
Commonsense question answering (QA) requires a model to grasp commonsense and factual knowledge to answer questions about world events. Many prior methods couple language modeling with knowledge graphs (KG). However, although a KG contains rich structural information, it lacks the context to provide a more precise understanding of the concepts. This creates a gap when fusing knowledge graphs into language modeling, especially when there is insufficient labeled data. Thus, we propose to employ external entity descriptions to provide contextual information for knowledge understanding. We retrieve descriptions of related concepts from Wiktionary and feed them as additional input to pre-trained language models. The resulting model achieves state-of-the-art result in the CommonsenseQA dataset and the best result among non-generative models in OpenBookQA. Our code is available at https://github.com/microsoft/DEKCOR-CommonsenseQA.
中文速览
常识问答（Commonsense QA）要求模型具备对日常世界的理解能力，但现有方法把语言模型和知识图谱（Knowledge Graph, KG）结合时，面临一个痛点：知识图谱虽然记录了概念之间的结构关系，却缺乏对每个概念含义的具体描述，导致模型难以真正理解概念的语义，在标注数据不足时尤为明显。为此，作者提出了DEKCOR模型，核心思路是从在线词典Wiktionary中检索相关概念的文字定义，将这些"描述性知识"与ConceptNet中提取的关系三元组一起拼接到ALBERT的输入中，让语言模型在做选择题时同时看到概念的结构关系和精确释义。实验表明，DEKCOR在CommonsenseQA数据集上超越此前最优结果1.2%（单模型）和3.8%（集成模型），在OpenBookQA上也在非生成式模型中取得最佳成绩。这项工作说明，给知识图谱中的节点补充自然语言描述是弥合结构知识与语言理解之间鸿沟的一种简单而有效的方式，对未来知识增强型问答系统的设计具有重要参考价值。
原文 arXiv:2012.04808;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2012.04808v3