A Few Brief Notes on DeepImpact, COIL, and a Conceptual Framework for Information Retrieval Techniques
Jimmy Lin Xueguang Ma David R. Cheriton School of Computer Science University of Waterloo
Abstract
Recent developments in representational learning for information retrieval can be organized in a conceptual framework that establishes two pairs of contrasts: sparse vs. dense representations and unsupervised vs. learned representations. Sparse learned representations can further be decomposed into expansion and term weighting components. This framework allows us to understand the relationship between recently proposed techniques such as DPR, ANCE, DeepCT, DeepImpact, and COIL, and furthermore, gaps revealed by our analysis point to “low hanging fruit” in terms of techniques that have yet to be explored. We present a novel technique dubbed “uniCOIL”, a simple extension of COIL that achieves to our knowledge the current state-of-the-art in sparse retrieval on the popular MS MARCO passage ranking dataset. Our implementation using the Anserini IR toolkit is built on the Lucene search library and thus fully compatible with standard inverted indexes.
中文速览
信息检索领域最近涌现出大量表示学习新方法，却缺乏统一的视角来厘清它们之间的关系。作者提出一个二维概念框架，从"稀疏/稠密表示"与"无监督/有监督学习"两个维度重新审视DPR、ANCE、DeepCT、DeepImpact、COIL等主流技术，并进一步把稀疏学习表示拆解为"文档扩展"和"词项权重"两个子模块，从而发现了一批尚未被探索的模型组合。基于这一分析，作者提出了名为uniCOIL的新方法——它是COIL的简化变体，将每个词的向量权重压缩为标量，并配合ReLU和8位量化使其能直接兼容标准倒排索引；在加入doc2query–T5文档扩展后，uniCOIL在MS MARCO段落排序任务上超越了DeepImpact，达到当时稀疏检索的最优水平，且索引体积仅1.3 GB，远小于COIL-tok的约40 GB。这项工作的意义在于：它不仅证明了神经网络并没有让稀疏表示过时，还为稀疏与稠密方法的互补融合提供了清晰的理论框架和可复现的工程基础。
原文 arXiv:2106.14807;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2106.14807v1