RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering

中文速览

开放域问答中的密集段落检索面临训练时负例太少、训练与实际检索场景不一致、未标注的真实答案段落被误当成负例，以及标注数据有限等问题。RocketQA采用跨批次负例、去噪困难负例和数据增强三种策略：扩大负例规模，过滤可能的假负例，并用交叉编码器给海量无标注数据生成伪标签来补充训练。实验表明，RocketQA在MSMARCO和Natural Questions上都显著超过此前最强方法，且能提升端到端问答效果。它说明通过更大、更可靠的训练信号，可以让高效的双编码器检索器更接近复杂真实检索场景，同时兼顾速度与准确率。

摘要

In open-domain question answering, dense passage retrieval has become a new paradigm to retrieve relevant passages for finding answers. Typically, the dual-encoder architecture is adopted to learn dense representations of questions and passages for semantic matching. However, it is difficult to effectively train a dual-encoder due to the challenges including the discrepancy between training and inference, the existence of unlabeled positives and limited training data. To address these challenges, we propose an optimized training approach, called RocketQA, to improving dense passage retrieval. We make three major technical contributions in RocketQA, namely cross-batch negatives, denoised hard negatives and data augmentation. The experiment results show that RocketQA significantly outperforms previous state-of-the-art models on both MSMARCO and Natural Questions. We also conduct extensive experiments to examine the effectiveness of the three strategies in RocketQA. Besides, we demonstrate that the performance of end-to-end QA can be improved based on our RocketQA retriever 11 1 Our code is available at https://github.com/PaddlePaddle/Research/tree/master/NLP/NAACL2021-RocketQA.

术语表

open-domain question answering (QA)

开放域问答（QA）

dense passage retrieval

密集段落检索

dual-encoder architecture

双编码器架构

cross-encoder architecture

交叉编码器架构

RocketQA

RocketQA

semantic matching

语义匹配

dense representation

稠密表示

embedding

嵌入向量

cross-batch negatives

跨批次负例

in-batch random negatives

批内随机负例

denoised hard negatives

去噪困难负例

hard negatives

困难负例

data augmentation

数据增强

unlabeled positives

未标注正例

false negatives

假负例

training and inference discrepancy

训练与推理不一致

pseudo labels

伪标签

end-to-end QA

端到端问答

machine reading comprehension (MRC)

机器阅读理解（MRC）

information retrieval (IR)

信息检索（IR）

TF-IDF

TF-IDF

BM25

BM25

inverted index

倒排索引

term mismatch

词项不匹配

passage retriever

段落检索器

neural reader

神经阅读器

MSMARCO

MSMARCO

Natural Questions

Natural Questions