Complement Lexical Retrieval Model with Semantic Residual Embeddings
Luyu Gao1 Zhuyun Dai1 Tongfei Chen2 Zhen Fan1 Benjamin Van Durme2 Jamie Callan1
Abstract
This paper presents clear, a retrieval model that seeks to complement classical lexical exact-match models such as BM25 with semantic matching signals from a neural embedding matching model. clear explicitly trains the neural embedding to encode language structures and semantics that lexical retrieval fails to capture with a novel residual-based embedding learning method. Empirical evaluations demonstrate the advantages of clear over state-of-the-art retrieval models, and that it can substantially improve the end-to-end accuracy and efficiency of reranking pipelines.
中文速览
现代搜索引擎的第一阶段检索长期依赖BM25这类基于词汇精确匹配的模型，这类模型在查询与文档用不同词汇表达同一概念时会严重失效。为此，研究者提出了CLEAR（Complementary Retrieval Model），其核心思路是用一个基于BERT的神经向量检索模型去"补足"BM25的不足：通过残差学习（residual-based learning）——即从BM25检索出错的案例中采样负样本、并用BM25分数差动态调整训练损失的边界——让神经模型专门学习那些词汇匹配捕捉不到的语义信号，而不是重复学习BM25已经处理好的部分。在大规模段落检索基准MS MARCO上，CLEAR在召回率和端到端重排精度上均显著超越单独的BM25、单独的BERT向量检索以及两者朴素融合的基线，同时还能降低下游重排阶段的计算开销。这项工作首次将词汇检索与神经向量检索进行联合训练，为构建兼顾精确匹配与语义理解的高效检索系统提供了新思路。
原文 arXiv:2004.13969;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2004.13969v3