Answering Science Exam Questions Using Query Reformulation with Background Knowledge
Ryan Musa†, Xiaoyan Wang§, Achille Fokoue†, Nicholas Mattei∗, Maria Chang†, Pavan Kapanipathi†, Bassem Makni†, Kartik Talamadupula†, Michael Witbrock† \AND†IBM Research \addrIBM T.J. Watson Research Center Yorktown Heights, NY 10598 USA \email{ramusa, achille, kapanipa, krtalamad, \email{maria.chang, \AND§ University of Illinois at Urbana-Champaign \addrDepartment of Computer Science Urbana, IL 61801 USA \AND∗ Tulane University \addrDepartment of Computer Science New Orleans, LA 70115 USA
Abstract
Open-domain question answering (QA) is an important problem in AI and NLP that is emerging as a bellwether for progress on the generalizability of AI methods and techniques. Much of the progress in open-domain QA systems has been realized through advances in information retrieval (IR) methods and corpus construction. In this paper, we focus on the recently introduced ARC Challenge dataset, which contains 2,590 multiple choice questions authored for grade-school science exams. These questions are selected to be the most challenging for current QA systems, and current state of the art performance is only slightly better than random chance. We present a system that reformulates a given question into queries that are used to retrieve supporting text from a large corpus of science-related text. Our rewriter is able to incorporate background knowledge from ConceptNet. In tandem with a generic textual entailment system trained on SciTail that identifies support in the retrieved results, our system outperforms several strong baselines on the end-to-end QA task despite only being trained to identify essential terms in the original source question. We use a generalizable decision methodology
中文速览
小学科学考试题难倒了主流AI——这类题目被专门筛选出来，让基于检索或统计的方法全都接近"蒙题"水平，当前最好系统的准确率也只有略高于随机的表现。针对这一困境，研究者提出了一套三段式流水线：先用神经序列标注模型把原题改写成更精炼的检索查询，同时引入ConceptNet中的常识背景知识来丰富查询内容；再用改写后的查询从大规模科学文本语料库中检索相关段落；最后用在SciTail数据集上训练的文本蕴含（textual entailment）模型判断哪个候选答案最受证据支持，并通过一套通用决策规则做出最终选择。实验表明，尽管改写模型本身只被训练来识别题目中的关键词，整套系统在ARC Challenge测试集上仍超越了多个强基线。这项工作揭示了查询改写、背景知识融合与文本蕴含三者协同作用的潜力，也为如何突破开放域科学问答的瓶颈提供了可复现的思路和经验分析。
原文 arXiv:1809.05726;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1809.05726v2