WorldTree: A Corpus of Explanation Graphs for Elementary Science Questions supporting Multi-Hop Inference
Abstract
Developing methods of automated inference that are able to provide users with compelling human-readable justifications for why the answer to a question is correct is critical for domains such as science and medicine, where user trust and detecting costly errors are limiting factors to adoption. One of the central barriers to training question answering models on explainable inference tasks is the lack of gold explanations to serve as training data. In this paper we present a corpus of explanations for standardized science exams, a recent challenge task for question answering. We manually construct a corpus of detailed explanations for nearly all publicly available standardized elementary science question (approximately 1,680 $3^{rd}$ through $5^{th}$ grade questions) and represent these as “explanation graphs” – sets of lexically overlapping sentences that describe how to arrive at the correct answer to a question through a combination of domain and world knowledge. We also provide an explanation-centered tablestore, a collection of semi-structured tables that contain the knowledge to construct these elementary science explanations. Together, these two knowledge resources map out a
中文速览
科学与医学等高风险领域迫切需要问答系统不仅能给出正确答案，还能提供让人信服的自然语言解释，而制约这一能力发展的关键瓶颈在于缺乏带有标注解释的训练数据。为此，研究团队手工构建了一个面向小学标准化科学考试的大规模语料库，为约1680道3至5年级试题提供详细的"解释图"（explanation graph）——即通过词汇重叠将多个句子串联起来、展示如何逐步推导出正确答案的结构化知识路径；与此同时，还配套发布了一个包含62张半结构化表格、近5000条知识条目的"表格知识库"（tablestore）。分析表明，回答并解释一道小学科学题平均需要聚合约6条独立知识，远超现有信息聚合方法普遍只能处理2条知识的上限，揭示了当前模型在多跳推理上的核心瓶颈。这套资源为可解释推理任务提供了兼具自由文本和结构化形式的训练数据，有望推动问答系统在解释生成与多步推理能力上取得实质性突破。
原文 arXiv:1802.03052;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1802.03052v1