arXiv:2305.16291 · 中英对照阅读
Voyager:一种基于大语言模型的开放式具身智能体
Voyager: An Open-Ended Embodied Agent with Large Language Models
中文速览
在没有固定终点的《我的世界》里,智能体怎样才能自己不断探索、学会新技能,并把学过的本领用到新任务上?研究者做出了 Voyager,让 GPT-4 根据当前状态安排循序渐进的挑战,把完成任务的代码存成可重复调用的技能,并依据游戏反馈、报错和自我检查反复修改代码。实验中,Voyager 找到的独特物品是其他方法的 3.3 倍,探索距离长 2.3 倍,解锁关键科技进度最快可达 15.3 倍;换到新世界后,它也能调用旧技能完成陌生任务。这个结果说明,大语言模型不只会临时规划行动,也能通过积累和组合技能持续学习,为打造能在开放环境中自主成长的智能体提供了新思路。
摘要
我们提出了 Voyager,这是首个 Minecraft 中的大语言模型驱动的具身终身学习智能体,它能在无需人工干预的情况下持续探索世界、习得多样技能并取得新发现。Voyager 由三个关键组件构成:1)最大限度促进探索的自动课程;2)不断扩充的可执行代码技能库,用于存储和检索复杂行为;3)一种新颖的迭代提示机制,将环境反馈、执行错误和自我验证纳入其中,以改进程序。Voyager 通过黑盒查询与 GPT-4 交互,从而无需对模型参数进行微调。Voyager 开发的技能具有时间延展性、可解释性和组合性,能够迅速累积智能体的能力,并缓解灾难性遗忘。实证结果表明,Voyager 展现出强大的上下文内终身学习能力,并在 Minecraft 游戏中表现出卓越的熟练度。与此前的最先进方法相比,它获得的独特物品数量多出 $3.3\times$,行进距离长出 $2.3\times$,解锁科技树关键里程碑的速度快达 $15.3\times$。Voyager 能够在新的 Minecraft 世界中利用习得的技能库,从零开始解决新任务,而其他技术则难以实现泛化。
We introduce Voyager, the first LLM-powered embodied lifelong learning agent in Minecraft that continuously explores the world, acquires diverse skills, and makes novel discoveries without human intervention. Voyager consists of three key components: 1) an automatic curriculum that maximizes exploration, 2) an ever-growing skill library of executable code for storing and retrieving complex behaviors, and 3) a new iterative prompting mechanism that incorporates environment feedback, execution errors, and self-verification for program improvement. Voyager interacts with GPT-4 via blackbox queries, which bypasses the need for model parameter fine-tuning. The skills developed by Voyager are temporally extended, interpretable, and compositional, which compounds the agent’s abilities rapidly and alleviates catastrophic forgetting. Empirically, Voyager shows strong in-context lifelong learning capability and exhibits exceptional proficiency in playing Minecraft. It obtains $3.3\times$ more unique items, travels $2.3\times$ longer distances, and unlocks key tech tree milestones up to $15.3\times$ faster than prior SOTA. Voyager is able to utilize the learned skill library in a new Minecraft world to solve novel tasks from scratch, while other techniques struggle to generalize.
术语表
- Voyager
- Voyager
- LLM-powered embodied lifelong learning agent
- 大语言模型驱动的具身终身学习智能体
- Minecraft
- Minecraft
- automatic curriculum
- 自动课程
- skill library
- 技能库
- iterative prompting mechanism
- 迭代提示机制
- environment feedback
- 环境反馈
- self-verification
- 自我验证
- blackbox queries
- 黑盒查询
- in-context lifelong learning
- 上下文内终身学习
- catastrophic forgetting
- 灾难性遗忘
- reinforcement learning (RL)
- 强化学习(RL)
- imitation learning
- 模仿学习
- large language model (LLM)
- 大语言模型(LLM)
- embodied agent
- 具身智能体
- procedurally generated 3D terrains
- 程序生成的三维地形
- tech tree
- 科技树
- action space
- 动作空间
- low-level motor commands
- 底层运动指令
- temporally extended actions
- 时间延展动作
- compositional actions
- 组合式动作
- in-context learning
- 上下文学习
- novelty search
- 新颖性搜索
- embedding
- 嵌入向量
- code interpreter
- 代码解释器
- generalization
- 泛化
- ReAct
- ReAct
- Reflexion
- Reflexion
- AutoGPT
- AutoGPT
- MineDojo
- MineDojo
- GPT-4
- GPT-4