Aha.
正在载入中英对照阅读…

arXiv:2312.10997 · 中英对照阅读

面向大语言模型的检索增强生成:综述

Retrieval-Augmented Generation for Large Language Models: A Survey

Yunfan Gao、Yun Xiong、Xinyu Gao、Kangxiang Jia、Jinliu Pan、Yuxi Bi、Yi Dai、Jiawei Sun、Meng Wang、Haofen Wang

中文速览

大语言模型会编造事实、知识也可能过时,因此研究者用检索增强生成(RAG)让模型先从外部资料中找信息,再据此回答。本文系统梳理了 RAG 从基础的“检索—作答”,到优化检索流程,再到可灵活组合模块的三阶段演进,并归纳检索、生成和信息整合中的关键方法。作者还汇总了覆盖 26 类任务、近 50 个数据集的评测资源,分析现有难题并提出后续研究方向。这样的整理能帮助读者看清 RAG 技术如何发展、怎样评估,也为构建更可靠、能及时更新知识的 AI 应用提供参考。

摘要

大语言模型(LLMs)展现出令人瞩目的能力,但也面临幻觉、知识过时以及推理过程不透明且无法追溯等挑战。检索增强生成(RAG)通过整合来自外部数据库的知识,成为一种很有前景的解决方案。这不仅提升了生成结果的准确性和可信度,尤其是在知识密集型任务中,还支持持续更新知识并整合特定领域的信息。RAG 将大语言模型(LLMs)的内在知识与外部数据库中海量且动态变化的知识库协同融合。本文对 RAG 范式的发展进行了全面综述,详细考察了朴素 RAG、高级 RAG 和模块化 RAG。本文还深入审视了 RAG 框架的三大基础,即检索、生成和增强技术,并重点介绍了各个关键组成部分中融入的先进技术,从而深入阐释 RAG 系统的进展。此外,本文还介绍了最新的评估框架和基准测试。最后,本文概述了当前面临的挑战,并指出未来研究与发展的潜在方向。11 1 相关资源见 https://github.com/Tongji-KGLLM/RAG-Survey 。

Large Language Models (LLMs) showcase impressive capabilities but encounter challenges like hallucination, outdated knowledge, and non-transparent, untraceable reasoning processes. Retrieval-Augmented Generation (RAG) has emerged as a promising solution by incorporating knowledge from external databases. This enhances the accuracy and credibility of the generation, particularly for knowledge-intensive tasks, and allows for continuous knowledge updates and integration of domain-specific information. RAG synergistically merges LLMs’ intrinsic knowledge with the vast, dynamic repositories of external databases. This comprehensive review paper offers a detailed examination of the progression of RAG paradigms, encompassing the Naive RAG, the Advanced RAG, and the Modular RAG. It meticulously scrutinizes the tripartite foundation of RAG frameworks, which includes the retrieval, the generation and the augmentation techniques. The paper highlights the state-of-the-art technologies embedded in each of these critical components, providing a profound understanding of the advancements in RAG systems. Furthermore, this paper introduces up-to-date evaluation framework and benchmark. At the end, this article delineates the challenges currently faced and points out prospective avenues for research and development 11 1 Resources are available at https://github.com/Tongji-KGLLM/RAG-Survey .

术语表

Large Language Models (LLMs)
大语言模型
Retrieval-Augmented Generation (RAG)
检索增强生成
hallucination
幻觉
external database
外部数据库
knowledge-intensive task
知识密集型任务
Naive RAG
朴素 RAG
Advanced RAG
高级 RAG
Modular RAG
模块化 RAG
retrieval
检索
generation
生成
augmentation
增强
evaluation framework
评估框架
benchmark
基准测试
semantic similarity
语义相似度
knowledge base
知识库
Transformer architecture
Transformer 架构
Pre-Training Model (PTM)
预训练模型
in-context learning (ICL)
上下文学习
inference stage
推理阶段
fine-tuning
微调
indexing
索引构建
document chunk
文档块
embedding model
嵌入模型
vector representation
向量表示
vector database
向量数据库
similarity search
相似性搜索
parametric knowledge
参数化知识
downstream task
下游任务