Hindsight: Posterior-guided training of retrievers for improved open-ended generation
Ashwin Paranjape, Omar Khattab, Christopher Potts, Matei Zaharia、Christopher D. Manning Stanford University
Abstract
Many text generation systems benefit from using a retriever to retrieve passages from a textual knowledge corpus (e.g., Wikipedia) and providing these passages as additional context to the generator. For open-ended generation tasks (like generating informative utterances in conversations) many varied passages may be equally relevant and we find that existing methods that jointly train the retriever and generator underperform: the retriever may not find relevant passages even amongst the top-10 and the generator may hence not learn a preference to ground its generated output in them. We propose using an additional guide retriever that is allowed to use the target output and “in hindsight” retrieve relevant passages during training. We model the guide retriever after the posterior distribution Q of passages given the input and the target output and train it jointly with the standard retriever and the generator by maximizing the evidence lower bound (ELBo) in expectation over Q. For informative conversations from the Wizard of Wikipedia dataset, with posterior-guided training, the retriever finds passages with higher relevance in the top-10 (23% relative improvement), the generator’s
中文速览
检索增强生成（retrieval-augmented generation）系统在开放式文本生成任务中面临一个核心难题：训练时检索器只根据输入去找相关段落，但在"一个问题对应多种合理回答"的场景下，真正支撑目标回答的段落往往淹没在众多"看起来相关"的段落里，检索器和生成器因此都很难得到有效的训练信号。为此，研究者提出了 Hindsight 训练框架，额外引入一个"后验引导检索器"（guide retriever），它在训练时被允许同时看到输入和目标输出，能够精准找到与目标答案真正相关的段落；整个系统通过最大化证据下界（ELBo）将后验检索器、标准检索器和生成器联合训练，使生成器专注于高质量段落、检索器向后验分布靠拢。在知识型对话数据集 Wizard of Wikipedia 上，该方法让检索器在 top-10 召回中找到目标段落的成功率提升了 23%，生成器对检索段落的依赖程度提升了 19%，端到端生成质量也提升了 6.4%，证明了"事后回望"式的后验引导对开放式生成任务的显著价值。
原文 arXiv:2110.07752;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2110.07752v2