Understanding BERT Rankers Under Distillation
Luyu Gao Carnegie Mellon University , Zhuyun Dai Carnegie Mellon University and Jamie Callan Carnegie Mellon University
Abstract
Deep language models such as BERT pre-trained on large corpus have given a huge performance boost to the state-of-the-art information retrieval ranking systems. Knowledge embedded in such models allows them to pick up complex matching signals between passages and queries. However, the high computation cost during inference limits their deployment in real-world search scenarios. In this paper, we study if and how the knowledge for search within BERT can be transferred to a smaller ranker through distillation. Our experiments demonstrate that it is crucial to use a proper distillation procedure, which produces up to nine times speedup while preserving the state-of-the-art performance.
中文速览
让规模庞大、效果顶尖的BERT排序模型跑得更快、更适合实际搜索系统部署，是这项工作要解决的核心难题。研究者系统探索了三种将BERT排序知识"蒸馏"（distillation）到小模型的方案：仅蒸馏排序知识（Ranker Distill）、先蒸馏语言模型知识再微调（LM Distill + Fine-tuning）、以及两步蒸馏同时迁移语言模型与排序两类知识（LM Distill + Ranker Distill）。实验在MS MARCO和TREC 2019 Deep Learning两个标准数据集上表明，只做排序蒸馏效果最差，而"先蒸馏语言模型知识、再蒸馏排序知识"的两步方案能让仅有4层的极小模型在保持与原版BERT相当排序精度的同时实现约9倍的推理加速。这一结果告诉开发者：在压缩BERT排序器时，显式保留通用语言建模知识是不可或缺的一步，为在工业级搜索场景中低成本部署高质量神经排序模型提供了清晰可行的路线图。
原文 arXiv:2007.11088;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2007.11088v1