Read + Verify: Machine Reading Comprehension with Unanswerable Questions
Minghao Hu1, Furu Wei2, Yuxing Peng1, Zhen Huang1, Nan Yang2, Dongsheng Li1 1College of Computer, National University of Defense Technology 2Microsoft Research Asia Contribution during internship at Microsoft Research Asia.
Abstract
Machine reading comprehension with unanswerable questions aims to abstain from answering when no answer can be inferred. In addition to extract answers, previous works usually predict an additional “no-answer” probability to detect unanswerable cases. However, they fail to validate the answerability of the question by verifying the legitimacy of the predicted answer. To address this problem, we propose a novel read-then-verify system, which not only utilizes a neural reader to extract candidate answers and produce no-answer probabilities, but also leverages an answer verifier to decide whether the predicted answer is entailed by the input snippets. Moreover, we introduce two auxiliary losses to help the reader better handle answer extraction as well as no-answer detection, and investigate three different architectures for the answer verifier. Our experiments on the SQuAD 2.0 dataset show that our system obtains a score of 74.2 F1 on test set, achieving state-of-the-art results at the time of submission (Aug. 28th, 2018).
中文速览
针对机器阅读理解中"无答案问题"的识别难题，现有系统虽然会预测一个"无答案"概率，却从不反过来核查已抽取的候选答案是否真的合理。本文提出了一套"先阅读、再验证"（read-then-verify）的两阶段框架：第一阶段由神经阅读器抽取候选答案并给出初步的无答案判断，第二阶段由专门设计的答案验证器对比答案所在句子与问题之间的语义蕴含关系，进一步确认答案是否站得住脚；此外还引入了两个辅助损失函数，分别独立优化答案抽取与无答案检测，避免二者相互干扰。在 SQuAD 2.0 基准测试上，该系统在测试集取得 74.2 F1 的成绩，达到了提交时的最优水平。这项工作的意义在于，它明确将"验证答案合法性"纳入阅读理解流程，为处理带有否定、反义词、实体替换等复杂语言现象的无答案问题提供了更可靠的解决思路。
原文 arXiv:1808.05759;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1808.05759v5