RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering
Yingqi Qu1, Yuchen Ding1, Jing Liu1, Kai Liu1, Ruiyang Ren2 Wayne Xin Zhao, Daxiang Dong1, Hua Wu1 and Haifeng Wang1 1Baidu Inc.; 2Gaoling School of Artificial Intelligence, Renmin University of China {quyingqi, dingyuchen, liujing46, liukai20, dongdaxiang, wu_hua, Corresponding authors. † The work was done when Ruiyang Ren was doing internship at Baidu.
Abstract
In open-domain question answering, dense passage retrieval has become a new paradigm to retrieve relevant passages for finding answers. Typically, the dual-encoder architecture is adopted to learn dense representations of questions and passages for semantic matching. However, it is difficult to effectively train a dual-encoder due to the challenges including the discrepancy between training and inference, the existence of unlabeled positives and limited training data. To address these challenges, we propose an optimized training approach, called RocketQA, to improving dense passage retrieval. We make three major technical contributions in RocketQA, namely cross-batch negatives, denoised hard negatives and data augmentation. The experiment results show that RocketQA significantly outperforms previous state-of-the-art models on both MSMARCO and Natural Questions. We also conduct extensive experiments to examine the effectiveness of the three strategies in RocketQA. Besides, we demonstrate that the performance of end-to-end QA can be improved based on our RocketQA retriever 111Our code is available at https://github.com/PaddlePaddle/Research/tree/master/NLP/NAACL2021-RocketQA.
中文速览
开放域问答中的密集段落检索面临训练时负例太少、训练与实际检索场景不一致、未标注的真实答案段落被误当成负例，以及标注数据有限等问题。RocketQA采用跨批次负例、去噪困难负例和数据增强三种策略：扩大负例规模，过滤可能的假负例，并用交叉编码器给海量无标注数据生成伪标签来补充训练。实验表明，RocketQA在MSMARCO和Natural Questions上都显著超过此前最强方法，且能提升端到端问答效果。它说明通过更大、更可靠的训练信号，可以让高效的双编码器检索器更接近复杂真实检索场景，同时兼顾速度与准确率。
原文 arXiv:2010.08191;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2010.08191v2