Tool Learning with Foundation Models
Yujia Qin1, Shengding Hu1, Yankai Lin2 , Weize Chen1, Ning Ding1, Ganqu Cui1, Zheni Zeng1, Xuanhe Zhou1, Yufei Huang1, Chaojun Xiao1, Chi Han3, Yi Ren Fung3, Yusheng Su1, Huadong Wang1, Cheng Qian1, Runchu Tian1, Kunlun Zhu8, Shihao Liang8, Xingyu Shen1, Bokai Xu1, Zhen Zhang1, Yining Ye1, Bowen Li1, Ziwei Tang5, Jing Yi1, Yuzhang Zhu1, Zhenning Dai1, Lan Yan1, Xin Cong1, Yaxi Lu1, Weilin Zhao1, Yuxiang Huang1, Junxi Yan1, Xu Han1, Xian Sun7, Dahai Li7, Jason Phang4, Cheng Yang5, Tongshuang Wu6, Heng Ji3, Guoliang Li1, Zhiyuan Liu1∗, Maosong Sun1∗ 1Tsinghua University, 2Renmin University of China, 3University of Illinois Urbana-Champaign, 4New York University, 5Beijing University of Posts and Telecommunications, 6Carnegie Mellon University, 7Zhihu Inc., 8ModelBest Inc. Corresponding authors.
Abstract
Humans possess an extraordinary ability to create and utilize tools, allowing them to overcome physical limitations and explore new frontiers. With the advent of recent powerful foundation models, artificial intelligence systems have the potential to be equally adept in tool use as humans. This paradigm, which is dubbed as tool learning with foundation models, combines the strengths of specialized tools and foundation models to achieve enhanced accuracy, efficiency, and automation in problem-solving. Despite its immense potential, there is still a lack of a comprehensive understanding of key challenges, opportunities, and future endeavors in this field. To this end, we present a systematic investigation and comprehensive review of tool learning in this paper. We first introduce the background of tool learning, including its cognitive origins, the paradigm shift of foundation models, and the complementary roles of tools and models. We recapitulate existing tool learning research and formulate a general tool learning framework: starting from understanding the user instruction, models should learn to decompose a complex task into several subtasks, dynamically adjust their plan through
中文速览
人类使用和创造工具的能力是智能的重要体现，但如何让AI系统同样掌握这种能力，一直缺乏系统性的研究框架。这篇综述提出了"基础模型工具学习（tool learning with foundation models）"这一范式，系统梳理了如何让GPT-4、ChatGPT等大型基础模型理解用户意图、分解复杂任务、调用合适的外部工具（如搜索引擎、计算器、代码执行器等）来完成真实世界的任务。作者构建了一个统一的工具学习框架，涵盖任务规划、工具理解、训练策略和泛化能力，并在18种代表性工具上进行了实验，验证了当前基础模型通过简单提示就能有效使用工具的潜力。这项工作的重要性在于，它首次对工具学习进行了全面系统的总结，厘清了安全可信、工具创造、个性化适配等关键开放问题，为未来构建更强大、更自主的AI系统指明了方向。
原文 arXiv:2304.08354;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2304.08354v3