QuAC : Question Answering in Context
Eunsol Choi★♡ He He★♢ Mohit Iyyer★♣† Mark Yatskar★† Wen-tau Yih† Yejin Choi♡† Percy Liang♢ Luke Zettlemoyer♡ Allen Institute for Artificial Intelligence† University of Washington♡ Stanford University♢ UMass Amherst♣
Abstract
We present QuAC, a dataset for Question Answering in Context that contains 14K information-seeking QA dialogs (100K questions in total). The dialogs involve two crowd workers: (1) a student who poses a sequence of freeform questions to learn as much as possible about a hidden Wikipedia text, and (2) a teacher who answers the questions by providing short excerpts from the text. QuAC introduces challenges not found in existing machine comprehension datasets: its questions are often more open-ended, unanswerable, or only meaningful within the dialog context, as we show in a detailed qualitative evaluation. We also report results for a number of reference models, including a recently state-of-the-art reading comprehension architecture extended to model dialog context. Our best model underperforms humans by 20 F1, suggesting that there is significant room for future work on this data. Dataset, baseline, and leaderboard available at http://quac.ai.
中文速览
对话式问答（Conversational QA）中，模型往往只需理解单轮问题就能作答，缺乏对真实信息探索型对话的挑战；为此，研究者构建了 QuAC 数据集，包含 14,000 段、共约 10 万条问答对话，其中一名工作者在不知晓文章内容的情况下连续向另一名工作者提问，后者从维基百科段落中截取文字作答。与 SQuAD 等传统阅读理解数据集相比，QuAC 的问题更开放、更依赖上下文指代，且约有一成问题根本无法从文中找到答案，极大提升了任务难度。实验表明，即便采用当时最先进的阅读理解模型并融入对话历史，其性能仍比人类低约 20 个 F1 分，说明现有模型在理解连续对话语境方面还有很大提升空间。这一数据集的价值在于推动模型从"回答孤立问题"迈向"在真实对话流中持续理解与追问"，更贴近实际的人机交互场景。
原文 arXiv:1808.07036;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1808.07036v3