QuaRel: A Dataset and Models for Answering Questions about Qualitative Relationships
Oyvind Tafjord, Peter Clark, Matt Gardner, Wen-tau Yih, Ashish Sabharwal Allen Institute for AI, Seattle, WA
Abstract
Many natural language questions require recognizing and reasoning with qualitative relationships (e.g., in science, economics, and medicine), but are challenging to answer with corpus-based methods. Qualitative modeling provides tools that support such reasoning, but the semantic parsing task of mapping questions into those models has formidable challenges. We present QuaRel, a dataset of diverse story questions involving qualitative relationships that characterize these challenges, and techniques that begin to address them. The dataset has 2771 questions relating 19 different types of quantities. For example, “Jenny observes that the robot vacuum cleaner moves slower on the living room carpet than on the bedroom carpet. Which carpet has more friction?” We contribute (1) a simple and flexible conceptual framework for representing these kinds of questions; (2) the QuaRel dataset, including logical forms, exemplifying the parsing challenges; and (3) two novel models for this task, built as extensions of type-constrained semantic parsing. The first of these models (called QuaSP+) significantly outperforms off-the-shelf tools on QuaRel. The second (QuaSP+Zero) demonstrates zero-shot ca
中文速览
自然语言中大量问题涉及定性关系推理（比如"摩擦力越大速度越慢"），但语料库检索和词语关联等主流方法在这类问题上表现很差，因为答案依赖逻辑推理而非事实查找。研究团队为此构建了 QuaRel 数据集，收录了 2771 道涵盖 19 种物理/自然属性的多选故事题，并为每道题标注了形式化逻辑表示，同时提出了一套"双世界比较"的概念框架，把问题统一表示为两个场景之间的定性对比。在此基础上，他们开发了两个模型：QuaSP+ 通过显式追踪"世界标签"显著超越了现成语义解析工具；QuaSP+Zero 则实现了零样本泛化，无需额外训练数据即可处理全新的定性关系，这是此前方法做不到的。这项工作为将自然语言与定性推理引擎对接提供了数据基础和可扩展的建模思路，对科学、医学、经济学等需要推理而非查询的应用场景具有重要价值。
原文 arXiv:1811.08048;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1811.08048v1