Multi-Stage Document Ranking with BERT
Rodrigo Nogueira,1 Wei Yang,2 Kyunghyun Cho,3,4,5,6 Jimmy Lin2 1 Tandon School of Engineering, New York University 2 David R. Cheriton School of Computer Science, University of Waterloo 3 Courant Institute of Mathematical Sciences, New York University 4 Center for Data Science, New York University 5 Facebook AI Research 6 CIFAR Azrieli Global Scholar
Abstract
The advent of deep neural networks pre-trained via language modeling tasks has spurred a number of successful applications in natural language processing. This work explores one such popular model, BERT, in the context of document ranking. We propose two variants, called monoBERT and duoBERT, that formulate the ranking problem as pointwise and pairwise classification, respectively. These two models are arranged in a multi-stage ranking architecture to form an end-to-end search system. One major advantage of this design is the ability to trade off quality against latency by controlling the admission of candidates into each pipeline stage, and by doing so, we are able to find operating points that offer a good balance between these two competing metrics. On two large-scale datasets, MS MARCO and TREC CAR, experiments show that our model produces results that are either at or comparable to the state of the art. Ablation studies show the contributions of each component and characterize the latency/quality tradeoff space.
中文速览
文档排序任务长期依赖人工特征，深度神经网络虽带来新思路，但单一模型在效果与推理速度之间难以兼顾。为此，研究者提出了基于BERT的多阶段排序框架，包含两个核心模块：monoBERT将每篇候选文档与查询独立配对、做二分类打分（逐点式），duoBERT则同时输入一对候选文档与查询、判断哪篇更相关（成对式），两者串联成流水线，前者粗筛、后者精排。通过控制每个阶段输入候选集的大小，系统可以灵活调节推理延迟与排序质量之间的平衡，在不显著增加延迟的前提下充分发挥duoBERT更强的判别能力。在MS MARCO和TREC CAR两个大规模数据集上，该系统达到或接近当时最优水平，消融实验也清晰验证了每个阶段各自的贡献。这项工作将信息检索领域成熟的多阶段级联思想与预训练大模型相结合，为高质量、可实用的神经文档检索系统提供了一套经过验证的设计范式。
原文 arXiv:1910.14424;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1910.14424v1