arXiv:2412.15266 · 中英对照阅读
LLM 智能体的结构化记忆
On the Structural Memory of LLM Agents
中文速览
面对长对话和复杂问答,智能体该怎样整理、查找记忆,才能既答得准又不被无关信息干扰?研究者把记忆整理成文本片段、知识关系、原子事实、摘要及其混合形式,并比较直接检索、重排和反复改写查询再检索等方法,在四类任务、六个数据集上进行测试。结果显示,不同记忆形式各有所长:片段和摘要更适合长文本与对话,知识关系和原子事实更利于精准推理,而混合记忆在不同任务间表现均衡、抗噪性也更强;反复检索在多数场景中效果最好。这项研究说明,智能体没有一种放之四海皆准的记忆方案,记忆结构和检索方式应结合任务来设计,也为搭建更可靠的智能体提供了实证依据。
摘要
记忆在使基于大语言模型(LLM)的智能体能够开展复杂的长期交互(如问答(QA)和对话系统)方面发挥着关键作用。尽管针对这些任务已提出多种记忆模块,但不同记忆结构在各类任务中的影响仍缺乏充分探索。本文研究记忆结构和记忆检索方法如何影响基于 LLM 的智能体的性能。具体而言,我们评估了四种记忆结构,包括文本块、知识三元组、原子事实和摘要,以及结合这些组件的混合记忆。此外,我们还评估了三种广泛使用的记忆检索方法:单步检索、重排序和迭代检索。在四项任务和六个数据集上开展的大量实验得出以下关键见解:(1)不同记忆结构各有优势,可针对特定任务进行调整;(2)混合记忆结构在噪声环境中表现出显著的抗干扰能力;(3)在各种场景下,迭代检索始终优于其他方法。我们的研究旨在启发人们进一步探索基于 LLM 的智能体的记忆系统设计。11 1 所有代码和数据集均已公开,网址为:https://github.com/zengrh3/StructuralMemory
Memory plays a pivotal role in enabling large language model (LLM)-based agents to engage in complex and long-term interactions, such as question answering (QA) and dialogue systems. While various memory modules have been proposed for these tasks, the impact of different memory structures across tasks remains insufficiently explored. This paper investigates how memory structures and memory retrieval methods affect the performance of LLM-based agents. Specifically, we evaluate four types of memory structures, including chunks, knowledge triples, atomic facts, and summaries, along with mixed memory that combines these components. In addition, we evaluate three widely used memory retrieval methods: single-step retrieval, reranking, and iterative retrieval. Extensive experiments conducted across four tasks and six datasets yield the following key insights: (1) Different memory structures offer distinct advantages, enabling them to be tailored to specific tasks; (2) Mixed memory structures demonstrate remarkable resilience in noisy environments; (3) Iterative retrieval consistently outperforms other methods across various scenarios. Our investigation aims to inspire further research into the design of memory systems for LLM-based agents. 11 1 All code and datasets are publicly available at: https://github.com/zengrh3/StructuralMemory
术语表
- large language model (LLM)
- 大语言模型(LLM)
- LLM-based agent
- 基于 LLM 的智能体
- memory module
- 记忆模块
- memory structure
- 记忆结构
- memory retrieval
- 记忆检索
- chunks
- 文本块
- knowledge triples
- 知识三元组
- atomic facts
- 原子事实
- summaries
- 摘要
- mixed memory
- 混合记忆
- single-step retrieval
- 单步检索
- reranking
- 重排序
- iterative retrieval
- 迭代检索
- multi-hop question answering (QA)
- 多跳问答(QA)
- single-hop QA
- 单跳问答
- dialogue understanding
- 对话理解
- reading comprehension
- 阅读理解
- retrieval-augmented generation (RAG)
- 检索增强生成(RAG)
- working memory
- 工作记忆
- episodic memory
- 情景记忆
- semantic memory
- 语义记忆
- noise robustness
- 抗噪能力
- relational reasoning
- 关系推理
- Artificial General Intelligence (AGI)
- 通用人工智能(AGI)
- HiAgent
- HiAgent
- Arigraph
- Arigraph
- ReadAgent
- ReadAgent
- GraphReader
- GraphReader