Aha.
正在载入中英对照阅读…

arXiv:2305.16291 · 中英对照阅读

Voyager:一种基于大语言模型的开放式具身智能体

Voyager: An Open-Ended Embodied Agent with Large Language Models

Guanzhi Wang、Yuqi Xie、Yunfan Jiang、Ajay Mandlekar、Chaowei Xiao、Yuke Zhu、Linxi “Jim” Fan、Anima Anandkumar、*Equal contribution †Equal advising Corresponding authorshttps://voyager.minedojo.org

中文速览

在没有固定终点的《我的世界》里,智能体怎样才能自己不断探索、学会新技能,并把学过的本领用到新任务上?研究者做出了 Voyager,让 GPT-4 根据当前状态安排循序渐进的挑战,把完成任务的代码存成可重复调用的技能,并依据游戏反馈、报错和自我检查反复修改代码。实验中,Voyager 找到的独特物品是其他方法的 3.3 倍,探索距离长 2.3 倍,解锁关键科技进度最快可达 15.3 倍;换到新世界后,它也能调用旧技能完成陌生任务。这个结果说明,大语言模型不只会临时规划行动,也能通过积累和组合技能持续学习,为打造能在开放环境中自主成长的智能体提供了新思路。

摘要

我们提出了 Voyager,这是首个 Minecraft 中的大语言模型驱动的具身终身学习智能体,它能在无需人工干预的情况下持续探索世界、习得多样技能并取得新发现。Voyager 由三个关键组件构成:1)最大限度促进探索的自动课程;2)不断扩充的可执行代码技能库,用于存储和检索复杂行为;3)一种新颖的迭代提示机制,将环境反馈、执行错误和自我验证纳入其中,以改进程序。Voyager 通过黑盒查询与 GPT-4 交互,从而无需对模型参数进行微调。Voyager 开发的技能具有时间延展性、可解释性和组合性,能够迅速累积智能体的能力,并缓解灾难性遗忘。实证结果表明,Voyager 展现出强大的上下文内终身学习能力,并在 Minecraft 游戏中表现出卓越的熟练度。与此前的最先进方法相比,它获得的独特物品数量多出 $3.3\times$,行进距离长出 $2.3\times$,解锁科技树关键里程碑的速度快达 $15.3\times$。Voyager 能够在新的 Minecraft 世界中利用习得的技能库,从零开始解决新任务,而其他技术则难以实现泛化。

We introduce Voyager, the first LLM-powered embodied lifelong learning agent in Minecraft that continuously explores the world, acquires diverse skills, and makes novel discoveries without human intervention. Voyager consists of three key components: 1) an automatic curriculum that maximizes exploration, 2) an ever-growing skill library of executable code for storing and retrieving complex behaviors, and 3) a new iterative prompting mechanism that incorporates environment feedback, execution errors, and self-verification for program improvement. Voyager interacts with GPT-4 via blackbox queries, which bypasses the need for model parameter fine-tuning. The skills developed by Voyager are temporally extended, interpretable, and compositional, which compounds the agent’s abilities rapidly and alleviates catastrophic forgetting. Empirically, Voyager shows strong in-context lifelong learning capability and exhibits exceptional proficiency in playing Minecraft. It obtains $3.3\times$ more unique items, travels $2.3\times$ longer distances, and unlocks key tech tree milestones up to $15.3\times$ faster than prior SOTA. Voyager is able to utilize the learned skill library in a new Minecraft world to solve novel tasks from scratch, while other techniques struggle to generalize.

术语表

Voyager
Voyager
LLM-powered embodied lifelong learning agent
大语言模型驱动的具身终身学习智能体
Minecraft
Minecraft
automatic curriculum
自动课程
skill library
技能库
iterative prompting mechanism
迭代提示机制
environment feedback
环境反馈
self-verification
自我验证
blackbox queries
黑盒查询
in-context lifelong learning
上下文内终身学习
catastrophic forgetting
灾难性遗忘
reinforcement learning (RL)
强化学习(RL)
imitation learning
模仿学习
large language model (LLM)
大语言模型(LLM)
embodied agent
具身智能体
procedurally generated 3D terrains
程序生成的三维地形
tech tree
科技树
action space
动作空间
low-level motor commands
底层运动指令
temporally extended actions
时间延展动作
compositional actions
组合式动作
in-context learning
上下文学习
novelty search
新颖性搜索
embedding
嵌入向量
code interpreter
代码解释器
generalization
泛化
ReAct
ReAct
Reflexion
Reflexion
AutoGPT
AutoGPT
MineDojo
MineDojo
GPT-4
GPT-4