arXiv:2312.10997 · 中英对照阅读
面向大语言模型的检索增强生成:综述
Retrieval-Augmented Generation for Large Language Models: A Survey
中文速览
大语言模型会编造事实、知识也可能过时,因此研究者用检索增强生成(RAG)让模型先从外部资料中找信息,再据此回答。本文系统梳理了 RAG 从基础的“检索—作答”,到优化检索流程,再到可灵活组合模块的三阶段演进,并归纳检索、生成和信息整合中的关键方法。作者还汇总了覆盖 26 类任务、近 50 个数据集的评测资源,分析现有难题并提出后续研究方向。这样的整理能帮助读者看清 RAG 技术如何发展、怎样评估,也为构建更可靠、能及时更新知识的 AI 应用提供参考。
摘要
大语言模型(LLMs)展现出令人瞩目的能力,但也面临幻觉、知识过时以及推理过程不透明且无法追溯等挑战。检索增强生成(RAG)通过整合来自外部数据库的知识,成为一种很有前景的解决方案。这不仅提升了生成结果的准确性和可信度,尤其是在知识密集型任务中,还支持持续更新知识并整合特定领域的信息。RAG 将大语言模型(LLMs)的内在知识与外部数据库中海量且动态变化的知识库协同融合。本文对 RAG 范式的发展进行了全面综述,详细考察了朴素 RAG、高级 RAG 和模块化 RAG。本文还深入审视了 RAG 框架的三大基础,即检索、生成和增强技术,并重点介绍了各个关键组成部分中融入的先进技术,从而深入阐释 RAG 系统的进展。此外,本文还介绍了最新的评估框架和基准测试。最后,本文概述了当前面临的挑战,并指出未来研究与发展的潜在方向。11 1 相关资源见 https://github.com/Tongji-KGLLM/RAG-Survey 。
Large Language Models (LLMs) showcase impressive capabilities but encounter challenges like hallucination, outdated knowledge, and non-transparent, untraceable reasoning processes. Retrieval-Augmented Generation (RAG) has emerged as a promising solution by incorporating knowledge from external databases. This enhances the accuracy and credibility of the generation, particularly for knowledge-intensive tasks, and allows for continuous knowledge updates and integration of domain-specific information. RAG synergistically merges LLMs’ intrinsic knowledge with the vast, dynamic repositories of external databases. This comprehensive review paper offers a detailed examination of the progression of RAG paradigms, encompassing the Naive RAG, the Advanced RAG, and the Modular RAG. It meticulously scrutinizes the tripartite foundation of RAG frameworks, which includes the retrieval, the generation and the augmentation techniques. The paper highlights the state-of-the-art technologies embedded in each of these critical components, providing a profound understanding of the advancements in RAG systems. Furthermore, this paper introduces up-to-date evaluation framework and benchmark. At the end, this article delineates the challenges currently faced and points out prospective avenues for research and development 11 1 Resources are available at https://github.com/Tongji-KGLLM/RAG-Survey .
术语表
- Large Language Models (LLMs)
- 大语言模型
- Retrieval-Augmented Generation (RAG)
- 检索增强生成
- hallucination
- 幻觉
- external database
- 外部数据库
- knowledge-intensive task
- 知识密集型任务
- Naive RAG
- 朴素 RAG
- Advanced RAG
- 高级 RAG
- Modular RAG
- 模块化 RAG
- retrieval
- 检索
- generation
- 生成
- augmentation
- 增强
- evaluation framework
- 评估框架
- benchmark
- 基准测试
- semantic similarity
- 语义相似度
- knowledge base
- 知识库
- Transformer architecture
- Transformer 架构
- Pre-Training Model (PTM)
- 预训练模型
- in-context learning (ICL)
- 上下文学习
- inference stage
- 推理阶段
- fine-tuning
- 微调
- indexing
- 索引构建
- document chunk
- 文档块
- embedding model
- 嵌入模型
- vector representation
- 向量表示
- vector database
- 向量数据库
- similarity search
- 相似性搜索
- parametric knowledge
- 参数化知识
- downstream task
- 下游任务