The Expando-Mono-Duo Design Pattern for Text Ranking with Pretrained Sequence-to-Sequence Models
Ronak Pradeep, Rodrigo Nogueira, and Jimmy Lin David R. Cheriton School of Computer Science, University of Waterloo
Abstract
We propose a design pattern for tackling text ranking problems, dubbed “Expando-Mono-Duo”, that has been empirically validated for a number of ad hoc retrieval tasks in different domains. At the core, our design relies on pretrained sequence-to-sequence models within a standard multi-stage ranking architecture. “Expando” refers to the use of document expansion techniques to enrich keyword representations of texts prior to inverted indexing. “Mono” and “Duo” refer to components in a reranking pipeline based on a pointwise model and a pairwise model that rerank initial candidates retrieved using keyword search. We present experimental results from the MS MARCO passage and document ranking tasks, the TREC 2020 Deep Learning Track, and the TREC-COVID challenge that validate our design. In all these tasks, we achieve effectiveness that is at or near the state of the art, in some cases using a zero-shot approach that does not exploit any training data from the target task. To support replicability, implementations of our design pattern are open-sourced in the Pyserini IR toolkit and PyGaggle neural reranking library.
中文速览
文本排序领域长期面临这样一个挑战：如何在不同任务、不同领域中都能稳定达到高质量的检索效果。研究者提出了一套名为"Expando-Mono-Duo"的设计模式，核心是将文档扩展（doc2query-T5，用序列到序列模型为文档预测潜在查询词并追加到原文）、BM25关键词检索、逐点重排序（monoT5）和逐对重排序（duoT5）四个环节串联成一条多阶段流水线，全程依赖预训练的T5序列到序列模型而非常见的BERT类编码器。在MS MARCO段落与文档排序、TREC 2020深度学习赛道以及TREC-COVID五项基准测试上，该方案的效果均达到或接近当时最优水平，部分场景甚至完全不使用目标任务的训练数据（零样本迁移）。这项工作的价值在于首次将几个此前零散发表的组件整合成一套完整、可复现的检索设计范式，并通过消融实验厘清了各组件的具体贡献，相关代码已在Pyserini和PyGaggle两个开源库中公开。
原文 arXiv:2101.05667;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2101.05667v1