Questions Are All You Need to Train a Dense Passage Retriever
Devendra Singh Sachan1,2, Mike Lewis3, Dani Yogatama4, Luke Zettlemoyer3,5, Joelle Pineau1,2,3, Manzil Zaheer4 1McGill University; 2Mila - Quebec AI Institute; 3Meta AI 4DeepMind; 5University of Washington {dyogatama,
Abstract
We introduce ART, a new corpus-level autoencoding approach for training dense retrieval models that does not require any labeled training data. Dense retrieval is a central challenge for open-domain tasks, such as Open QA, where state-of-the-art methods typically require large supervised datasets with custom hard-negative mining and denoising of positive examples. ART, in contrast, only requires access to unpaired inputs and outputs (e.g. questions and potential answer passages). It uses a new passage-retrieval autoencoding scheme, where (1) an input question is used to retrieve a set of evidence passages, and (2) the passages are then used to compute the probability of reconstructing the original question. Training for retrieval based on question reconstruction enables effective unsupervised learning of both passage and question encoders, which can be later incorporated into complete Open QA systems without any further finetuning. Extensive experiments demonstrate that ART obtains state-of-the-art results on multiple QA retrieval benchmarks with only generic initialization from a pre-trained language model, removing the need for labeled data and task-specific losses.111Our code an
中文速览
开放域问答（Open-domain QA）系统的核心难题之一，是如何从海量文章中精准检索出与问题相关的段落，而现有最强的密集检索（dense retrieval）方法都依赖大量人工标注的问题-段落配对数据。ART（Autoencoding-based Retriever Training）提出了一种完全无监督的训练框架：给定一个问题，先用双编码器检索一批候选段落，再用一个冻结的大型预训练语言模型（如 T5）计算"用该段落能还原出原始问题的概率"，以此作为软标签来监督检索器的训练——整个过程无需任何人工标注的问题-段落对。在五个主流问答基准上，ART 的检索准确率平均比经典有监督方法 DPR 高出约 4–5 个百分点，甚至仅凭 1000 个无标注问题就能超越 DPR，同时具备良好的跨数据集泛化能力。这项工作证明，借助强大预训练模型的零样本推理能力，可以绕过昂贵的人工标注流程，为低资源场景下构建高质量检索系统提供了切实可行的新路径。
原文 arXiv:2206.10658;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2206.10658v4