Phrase-Indexed Question Answering: A New Challenge for Scalable Document Comprehension
Minjoon Seo2,3 Tom Kwiatkowski1 Ankur P. Parikh1 Ali Farhadi2,4,5 Hannaneh Hajishirzi2 Google AI Language1 University of Washington2 Clova AI, NAVER3 Allen Institute for AI4 XNOR.AI5 Most work done during internship with Google AI.
Abstract
We formalize a new modular variant of current question answering tasks by enforcing complete independence of the document encoder from the question encoder. This formulation addresses a key challenge in machine comprehension by requiring a standalone representation of the document discourse. It additionally leads to a significant scalability advantage since the encoding of the answer candidate phrases in the document can be pre-computed and indexed offline for efficient retrieval. We experiment with baseline models for the new task, which achieve a reasonable accuracy but significantly underperform unconstrained QA models. We invite the QA research community to engage in Phrase-Indexed Question Answering (PIQA, pika) for closing the gap. The leaderboard is at: nlp.cs.washington.edu/piqa
中文速览
现有的抽取式问答模型在处理文档时必须"看着问题来读文章"，每来一个新问题就得把整篇文档重新过一遍神经网络，既无法独立评估模型对文档的真实理解能力，也难以大规模部署。为此，研究者提出了"短语索引问答"（Phrase-Indexed Question Answering，PIQA）这一新范式：强制要求文档编码器与问题编码器完全独立，文档中每个候选答案短语被预先编码成向量并离线建索引，回答问题时只需将问题向量与索引向量做内积检索，无需重新处理文档。在SQuAD上的基线实验表明，加入自注意力和ELMo的模型能达到尚可的准确率，且学到的短语向量确实捕捉到了有意义的语义和上下文信息，但与不受约束的最优模型相比仍有约26.6%的F1差距。这项工作的意义在于：它既为评估模型真正的文档理解能力提供了一个更严格的基准，又为构建可实际落地、支持亚线性时间近似检索的大规模问答系统指明了方向。
原文 arXiv:1804.07726;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1804.07726v2