GitAgent: Facilitating Autonomous Agent with GitHub by Tool Extension
Bohan Lyu1 , Xin Cong1∗ , Heyang Yu1, Pan Yang1, Yujia Qin1, Yining Ye1, Yaxi Lu1 Zhong Zhang1, Yukun Yan1, Yankai Lin2, Zhiyuan Liu1†, Maosong Sun1 1Tsinghua University 2Renmin University of China Indicates equal contribution. Corresponding author.
Abstract
While Large Language Models (LLMs) like ChatGPT and GPT-4 have demonstrated exceptional proficiency in natural language processing, their efficacy in addressing complex, multifaceted tasks remains limited. A growing area of research focuses on LLM-based agents equipped with external tools capable of performing diverse tasks. However, existing LLM-based agents only support a limited set of tools which is unable to cover a diverse range of user queries, especially for those involving expertise domains. It remains a challenge for LLM-based agents to extend their tools autonomously when confronted with various user queries. As GitHub has hosted a multitude of repositories which can be seen as a good resource for tools, a promising solution is that LLM-based agents can autonomously integrate the repositories in GitHub according to the user queries to extend their tool set. In this paper, we introduce GitAgent, an agent capable of achieving the autonomous tool extension from GitHub. GitAgent follows a four-phase procedure to incorporate repositories and it can learn human experience by resorting to GitHub Issues/PRs to solve problems encountered during the procedure. Experimental evaluat
中文速览
大型语言模型（LLM）虽然在自然语言处理上表现出色，但面对需要专业工具的复杂任务时往往力不从心，而现有的基于LLM的智能体所能调用的工具集又十分有限，难以覆盖量化投资、分子逆合成规划等专业领域的用户需求。为此，研究者提出了GitAgent——一个能够自主从GitHub上扩展工具的智能体，它将整个工具集成过程拆解为搜索、配置、应用、存储四个阶段，并通过读取GitHub上的Issues和Pull Requests来学习人类开发者积累的调试和使用经验，从而解决代码缺陷、文档不完整等现实问题。在涵盖四个专业领域（量化投资、老照片修复、基因结构变异检测、化学逆合成）共30条用户查询的实验中，GitAgent平均达到了69.4%的任务成功率。这项工作的重要意义在于，它为LLM智能体突破固定工具集的限制、按需自主扩展能力边界提供了一条可行路径，也为未来如何让AI更好地利用开源社区资源指明了方向。
原文 arXiv:2312.17294;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2312.17294v3