ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Cases
Qiaoyu Tang1,3, Ziliang Deng1,3, Hongyu Lin1, Xianpei Han1,, Qiao Liang1,3, Boxi Cao1,3, Le Sun1,2 Corresponding Authors
Abstract
Enabling large language models to utilize real-world tools effectively is crucial for achieving embodied intelligence. Existing approaches to tool learning have either primarily relied on extremely large language models, such as GPT-4, to attain generalized tool-use abilities in a zero-shot manner, or utilized supervised learning to train limited scopes of tools on compact models. However, it remains uncertain whether smaller language models can achieve generalized tool-use abilities without tool-specific training. To address this question, this paper introduces ToolAlpaca, a novel framework designed to automatically generate a diverse tool-use corpus and learn generalized tool-use abilities on compact language models with minimal human intervention. Specifically, ToolAlpaca first automatically creates a highly diversified tool-use corpus by building a multi-agent simulation environment. The corpus contains 3938 tool-use instances from more than 400 real-world tool APIs spanning 50 distinct categories. Subsequently, the constructed corpus is employed to fine-tune compact language models, resulting in two models, namely ToolAlpaca-7B and ToolAlpaca-13B, respectively. Finally, we eva
中文速览
让大型语言模型（LLM）学会使用现实世界中的各种工具，是迈向具身智能的关键一步，但目前要么依赖 GPT-4 这类超大模型的零样本能力，要么只能针对少数特定工具对小模型做监督微调，小型模型能否泛化到从未见过的工具始终是个未解之谜。为此，研究者提出了 ToolAlpaca 框架：先从互联网收集 400 余个真实 API 的简介，再用 LLM 自动补全结构化文档，然后构建一个由"用户、助手、工具执行者"三类虚拟智能体组成的多轮对话仿真环境，全自动生成覆盖 50 个类别、共 3938 条工具使用样例的多样化语料库，最后用该语料库微调 7B 和 13B 的紧凑模型。实验结果显示，经过微调的 ToolAlpaca 模型在应对未曾训练过的新工具时，性能与 GPT-3.5 相当，证明了小模型同样可以习得泛化的工具使用能力，这对降低具身智能的落地门槛具有重要意义。
原文 arXiv:2306.05301;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2306.05301v2