Think-in-Memory: Recalling and Post-thinking Enable LLMs with Long-Term Memory
Lei Liu 0000-0001-8109-5248 CUHK-Shenzhen, Ant Group , Xiaoyan Yang Ant Group , Yue Shen Ant Group , Binbin Hu, Zhiqiang Zhang bin.hbb, Ant Group , Jinjie Gu Ant Group and Guannan Zhang Ant Group
Abstract
Memory-augmented Large Language Models (LLMs) have demonstrated remarkable performance in long-term human-machine interactions, which basically relies on iterative recalling and reasoning of history to generate high-quality responses. However, such repeated recall-reason steps easily produce biased thoughts, i.e., inconsistent reasoning results when recalling the same history for different questions. On the contrary, humans can keep thoughts in the memory and recall them without repeated reasoning. Motivated by this human capability, we propose a novel memory mechanism called TiM (Think-in-Memory) that enables LLMs to maintain an evolved memory for storing historical thoughts along the conversation stream. The TiM framework consists of two crucial stages: (1) before generating a response, a LLM agent recalls relevant thoughts from memory, and (2) after generating a response, the LLM agent post-thinks and incorporates both historical and new thoughts to update the memory. Thus, TiM can eliminate the issue of repeated reasoning by saving the post-thinking thoughts as the history. Besides, we formulate the basic principles to organize the thoughts in memory based on the well-establish
中文速览
大语言模型（LLM）在长对话场景中存在一个根本缺陷：每次回答新问题时都要重新理解同一段历史记录，这种反复推理极易产生前后矛盾的结论。受人类"把想法本身存进记忆、而不是反复回忆原始事件"的元认知机制启发，研究者提出了 TiM（Think-in-Memory，记忆内思考）框架：LLM 在回答问题前先从外部记忆库中检索相关"思维摘要"，回答后再对本轮对话进行后思考，将新生成的归纳性思维以插入、遗忘、合并等操作动态更新到记忆库中，并引入局部敏感哈希（Locality-Sensitive Hashing）大幅提升长对话场景下的检索效率。在涵盖中英双语、开放域与专业领域的真实及模拟多轮对话数据集上，TiM 显著提升了 ChatGPT、ChatGLM 等多种 LLM 的回复准确性与一致性，且无需修改模型结构即可即插即用。这一工作为构建真正具备长期记忆能力的对话 AI 提供了一条低成本、高兼容性的新路径。
原文 arXiv:2311.08719;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2311.08719v1