Distilling Dense Representations for Ranking using Tightly-Coupled Teachers
Sheng-Chieh Lin Contributed equally. David R. Cheriton School of Computer Science University of Waterloo Jheng-Hong Yang David R. Cheriton School of Computer Science University of Waterloo Jimmy Lin David R. Cheriton School of Computer Science University of Waterloo
Abstract
We present an approach to ranking with dense representations that applies knowledge distillation to improve the recently proposed late-interaction ColBERT model. Specifically, we distill the knowledge from ColBERT’s expressive MaxSim operator for computing relevance scores into a simple dot product, thus enabling single-step ANN search. Our key insight is that during distillation, tight coupling between the teacher model and the student model enables more flexible distillation strategies and yields better learned representations. We empirically show that our approach improves query latency and greatly reduces the onerous storage requirements of ColBERT, while only making modest sacrifices in terms of effectiveness. By combining our dense representations with sparse representations derived from document expansion, we are able to approach the effectiveness of a standard cross-encoder reranker using BERT that is orders of magnitude slower.
中文速览
稠密向量检索（dense retrieval）虽然速度快，但效果往往不如计算昂贵的交叉编码器（cross-encoder）重排序模型，而 ColBERT 提出的"后期交互"机制虽能提升效果，却因存储每个词元的向量而占用大量空间、且无法直接进行单步近似最近邻搜索。本文提出用知识蒸馏（knowledge distillation）的方法，将 ColBERT 复杂的 MaxSim 相关性计算"压缩"进一个简单的点积双编码器（bi-encoder），其核心创新在于训练时让教师模型（ColBERT）与学生模型实时紧耦合、同步推理，而非提前预计算相关性分数，从而可以更灵活地探索难负样本组合，同时也省去了周期性刷新索引的巨大开销。实验在 MS MARCO 和 TREC 2019 Deep Learning 两个数据集上表明，所得到的 TCT-ColBERT 模型在显著降低查询延迟、大幅缩减存储占用的同时，检索效果损失极小；进一步将稠密表示与基于文档扩展的稀疏表示线性融合后，其效果已能接近比它慢几个数量级的 BERT 交叉编码器重排序器。这项工作为在工业级规模上同时兼顾效率与效果的端到端段落检索提供了一条切实可行的路径。
原文 arXiv:2010.11386;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2010.11386v1