Middleware for LLMs: Tools Are Instrumental for Language Agents in Complex Environments
Yu Gu1, Yiheng Shu1, Hao Yu2, Xiao Liu2, Yuxiao Dong2, Jie Tang2, Jayanth Srinivasa3, Hugo Latapie3, Yu Su1 1The Ohio State University 2Tsinghua University 3Cisco Research {gu.826,
Abstract
The applications of large language models (LLMs) have expanded well beyond the confines of text processing, signaling a new era where LLMs are envisioned as generalist agents capable of operating within complex environments. These environments are often highly expansive, making it impossible for the LLM to process them within its short-term memory. Motivated by recent research on extending the capabilities of LLMs with tools, we seek to investigate the intriguing potential of tools to augment LLMs in handling such complexity by introducing a novel class of tools, termed middleware, to aid in the proactive exploration within these massive environments. Such specialized tools can serve as a middleware layer shielding the LLM from environmental complexity. In two representative complex environments—knowledge bases (KBs) and databases—we demonstrate the significant potential of augmenting language agents with tools in complex environments. Notably, equipped with the middleware, GPT-4 achieves 2.8 $\times$ the performance of the best baseline in tasks requiring access to database content and 2.2 $\times$ in KB tasks. Our findings illuminate the path for advancing language agents in real
中文速览
大型语言模型（LLM）在处理知识库、数据库等"巨型复杂环境"时面临核心瓶颈：这些环境的信息量远超模型的上下文窗口，无法直接塞进去处理。研究者提出为LLM配备一套名为"中间件（middleware）"的定制化工具集——分别针对数据库和知识库手工设计了共十余种导航与功能工具，使模型能像人类一样主动探索环境、按需取用信息，而无需一次性加载全部内容；同时引入"错误反馈"和"解耦生成"两种新机制，让模型的推理步骤与工具调用更精准地协同。实验结果显示，配备中间件的GPT-4在数据库内容访问任务上达到最佳基线2.8倍的性能，在知识库问答任务上达到2.2倍。这项工作表明，为LLM量身打造领域专属工具是突破复杂现实环境瓶颈、迈向通用智能体的关键路径。
原文 arXiv:2402.14672;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2402.14672v2