Relevance-guided Supervision for OpenQA with ColBERT
Omar Khattab Stanford University、Christopher Potts Stanford University、Matei Zaharia Stanford University
Abstract
Systems for Open-Domain Question Answering (OpenQA) generally depend on a retriever for finding candidate passages in a large corpus and a reader for extracting answers from those passages. In much recent work, the retriever is a learned component that uses coarse-grained vector representations of questions and passages. We argue that this modeling choice is insufficiently expressive for dealing with the complexity of natural language questions. To address this, we define ColBERT-QA, which adapts the scalable neural retrieval model ColBERT to OpenQA. ColBERT creates fine-grained interactions between questions and passages. We propose an efficient weak supervision strategy that iteratively uses ColBERT to create its own training data. This greatly improves OpenQA retrieval on Natural Questions, SQuAD, and TriviaQA, and the resulting system attains state-of-the-art extractive OpenQA performance on all three datasets.
中文速览
开放域问答（OpenQA）系统需要先从海量文本中检索相关段落，再从中抽取答案，而现有检索器普遍把问题和段落各压缩成一个向量、用一次点积估算相关性，这种粗粒度方式难以应对复杂自然语言问题。为此，论文提出 ColBERT-QA，将能在词语级别做细粒度交互匹配的检索模型 ColBERT 引入 OpenQA，并设计了"相关性引导监督"（Relevance-Guided Supervision，RGS）策略——从弱检索器（如 BM25）出发，迭代地让检索器为自己生成训练数据，整个过程只需对语料库重新索引一两次，无需冻结文档编码器也无需频繁重建索引。在 Natural Questions、SQuAD 和 TriviaQA 三个基准上，ColBERT-QA 的检索准确率和端到端抽取式问答性能均达到当时最优，证明细粒度词级交互配合高效弱监督是提升 OpenQA 检索质量的有效路径。
原文 arXiv:2007.00814;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2007.00814v2