Adaptive Document Retrieval for Deep Question Answering
Bernhard Kratzwald and Stefan Feuerriegel Chair of Management Information Systems ETH Zurich Zurich, Switzerland {bkratzwald,
Abstract
State-of-the-art systems in deep question answering proceed as follows: (1) an initial document retrieval selects relevant documents, which (2) are then processed by a neural network in order to extract the final answer. Yet the exact interplay between both components is poorly understood, especially concerning the number of candidate documents that should be retrieved. We show that choosing a static number of documents – as used in prior research – suffers from a noise-information trade-off and yields suboptimal results. As a remedy, we propose an adaptive document retrieval model. This learns the optimal candidate number for document retrieval, conditional on the size of the corpus and the query. We report extensive experimental results showing that our adaptive approach outperforms state-of-the-art methods on multiple benchmark datasets, as well as in the context of corpora with variable sizes.
中文速览
现有的深度问答系统在检索文档时，都用一个固定的候选文档数量（比如取前5篇），但这个"一刀切"的做法忽略了一个根本矛盾：取的文档越多，召回率越高，但引入的噪声也越多，反而会干扰答案抽取模块。研究者通过实验揭示了这一"噪声-信息权衡"（noise-information trade-off）现象，并发现语料库规模的变化会使最优文档数量大幅波动——小语料库里取1篇最好，大语料库里取10篇才合适。为此，他们提出了一种自适应文档检索模型，包括基于置信度阈值的启发式方法和可训练的序数回归（ordinal regression）分类器，能够根据查询特征和语料规模动态决定每次检索应返回多少篇文档。在SQuAD、TREC、WikiMovies、WebQuestions四个主流基准上，自适应方法在三个数据集上超越了现有最优系统，并在所有数据集上均优于固定数量的基线，这为构建能适应动态增长语料库的鲁棒问答系统提供了切实可行的思路。
原文 arXiv:1808.06528;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1808.06528v1