HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face
Yongliang Shen1,2,∗, Kaitao Song2,∗,†, Xu Tan2, Dongsheng Li2, Weiming Lu1,†, Yueting Zhuang1,† Zhejiang University1, Microsoft Research Asia2 {syl, luwm, {kaitaosong, xuta, https://github.com/microsoft/JARVIS
Abstract
Solving complicated AI tasks with different domains and modalities is a key step toward artificial general intelligence. While there are numerous AI models available for various domains and modalities, they cannot handle complicated AI tasks autonomously. Considering large language models (LLMs) have exhibited exceptional abilities in language understanding, generation, interaction, and reasoning, we advocate that LLMs could act as a controller to manage existing AI models to solve complicated AI tasks, with language serving as a generic interface to empower this. Based on this philosophy, we present HuggingGPT, an LLM-powered agent that leverages LLMs (e.g., ChatGPT) to connect various AI models in machine learning communities (e.g., Hugging Face) to solve AI tasks. Specifically, we use ChatGPT to conduct task planning when receiving a user request, select models according to their function descriptions available in Hugging Face, execute each subtask with the selected AI model, and summarize the response according to the execution results. By leveraging the strong language capability of ChatGPT and abundant AI models in Hugging Face, HuggingGPT can tackle a wide range of sophistic
中文速览
大语言模型（LLM）虽然擅长理解和推理语言，却无法直接处理图像、音频等多模态任务，也难以独立应对需要多个专业模型协作的复杂任务。为此，研究者提出了 HuggingGPT，让 ChatGPT 充当"大脑"控制器，通过自然语言作为通用接口，自动调用 Hugging Face 平台上海量的专业 AI 模型来完成任务：整个流程分为任务规划、模型选择、任务执行和结果汇总四个阶段，ChatGPT 负责拆解用户需求、按模型描述挑选合适的专家模型、整合各模型的输出并生成最终回答。实验表明，HuggingGPT 能够跨越语言、视觉、语音及跨模态等多个领域完成复杂任务，效果令人印象深刻。这项工作的意义在于，它提供了一条无需重新训练模型、只靠语言接口就能持续扩展 AI 能力边界的新路径，朝着通用人工智能（AGI）迈进了重要一步。
原文 arXiv:2303.17580;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2303.17580v4