Efficient Retrieval Augmented Generation from Unstructured Knowledge for Task-Oriented Dialog
David Thulke,1,2 Nico Daheim,1 Christian Dugast,1,2 Hermann Ney1,2
Abstract
This paper summarizes our work on the first track of the ninth Dialog System Technology Challenge (DSTC 9), “Beyond Domain APIs: Task-oriented Conversational Modeling with Unstructured Knowledge Access”. The goal of the task is to generate responses to user turns in a task-oriented dialog that require knowledge from unstructured documents. The task is divided into three subtasks: detection, selection and generation. In order to be compute efficient, we formulate the selection problem in terms of hierarchical classification steps. We achieve our best results with this model. Alternatively, we employ siamese sequence embedding models, referred to as Dense Knowledge Retrieval, to retrieve relevant documents. This method further reduces the computation time by a factor of more than 100x at the cost of degradation in R@1 of 5-6% compared to the first model. Then for either approach, we use Retrieval Augmented Generation to generate responses based on multiple selected snippets and we show how the method can be used to fine-tune trained embeddings.
中文速览
面向任务的对话系统通常依赖结构化数据库接口来回答用户问题，但大量有用信息散落在网页描述、FAQ等非结构化文档中，如何高效地从这些文档里检索并生成合适回复是一大难题。针对DSTC 9第一赛道，作者提出了两种知识检索方案：一是将文档检索拆分为"先定位领域和实体、再精选片段"的层级分类流程，在显著降低计算量的同时还提升了检索准确率；二是用孪生网络（Siamese Network）学习对话与文档的稠密向量表示，通过向量相似度检索，速度比基线快逾100倍，召回率仅下降约5–6%。在生成阶段，作者引入检索增强生成（Retrieval Augmented Generation）框架，让模型同时参考多个候选文档片段生成回复，并可反向微调检索用的向量表示。这项工作为在真实对话场景中兼顾检索精度与响应速度提供了切实可行的解决思路。
原文 arXiv:2102.04643;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2102.04643v1