From Summary to Action: Enhancing Large Language Models for Complex Tasks with Open World APIs
Yulong Liu1 Yunlong Yuan2∗∗{}^{\ast}start_FLOATSUPERSCRIPT ∗ end_FLOATSUPERSCRIPT Chunwei Wang3 Jianhua Han3 Yongqiang Ma1 Li Zhang2 Nanning Zheng1 Hang Xu3 1National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Xi’an, China 2 School of Data Science, Fudan University 3 Huawei Noah’s Ark Lab Equal contributionCorresponding author
Abstract
The distinction between humans and animals lies in the unique ability of humans to use and create tools. Tools empower humans to overcome physiological limitations, fostering the creation of magnificent civilizations. Similarly, enabling foundational models like Large Language Models (LLMs) with the capacity to learn external tool usage may serve as a pivotal step toward realizing artificial general intelligence. Previous studies in this field have predominantly pursued two distinct approaches to augment the tool invocation capabilities of LLMs. The first approach emphasizes the construction of relevant datasets for model fine-tuning. The second approach, in contrast, aims to fully exploit the inherent reasoning abilities of LLMs through in-context learning strategies. In this work, we introduce a novel tool invocation pipeline designed to control massive real-world APIs. This pipeline mirrors the human task-solving process, addressing complicated real-life user queries. At each step, we guide LLMs to summarize the achieved results and determine the next course of action. We term this pipeline ‘from Summary to action’, Sum2Act for short. Empirical evaluations of our Sum2Act pipelin
中文速览
让大语言模型（LLM）像人一样调用海量真实世界API来完成复杂任务，是迈向通用人工智能的关键一步，但现有方法要么依赖大量微调数据，要么在推理时无法有效利用失败路径的信息。为此，研究者提出了Sum2Act框架：每执行一步工具调用后，系统的"状态管理器"都会对历史行动结果进行显式总结，记录已获得的进展和失败原因，再由"路由器"基于这份动态摘要决定下一步行动，从而模拟人类"边做边总结、遇阻换路"的工作方式。在包含逾1.6万个真实API的ToolBench基准上，Sum2Act显著超越了ReAct和DFSDT等主流方法，且无需额外微调即可扩展到视觉类API任务。这项工作表明，让模型在每一步都保持对任务状态的清晰认知，是提升LLM工具调用能力的有效途径。
原文 arXiv:2402.18157;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2402.18157v1