Lifelong Robot Learning with Human Assisted Language Planners
Meenal Parakh∗,α,γ, Alisha Fong∗,α,γ, Anthony Simeonovα,γ, Tao Chenα,γ, Abhishek Guptaα,β,γ, Pulkit Agrawalα,γ αImprobable AI Lab βUniversity of Washington γMassachusetts Institute of Technology ∗Authors contributed equally
Abstract
Large Language Models (LLMs) have been shown to act like planners that can decompose high-level instructions into a sequence of executable instructions. However, current LLM-based planners are only able to operate with a fixed set of skills. We overcome this critical limitation and present a method for using LLM-based planners to query new skills and teach robots these skills in a data and time-efficient manner for rigid object manipulation. Our system can re-use newly acquired skills for future tasks, demonstrating the potential of open world and lifelong learning. We evaluate the proposed framework on multiple tasks in simulation and the real world. Videos are available at: https://sites.google.com/mit.edu/halp-robot-learning
中文速览
大语言模型（LLM）可以把"清理水槽"这类高层指令拆解成一系列可执行步骤，但现有系统只能调用固定的技能库，一旦遇到库里没有的动作就束手无策。这项工作提出了一个名为 HALP 的框架，让 LLM 规划器在发现现有技能不够用时，主动调用一个特殊的 `learn_skill` 函数来申请新技能，再通过神经描述场（Neural Descriptor Fields，NDF）仅用 5–10 次人工示范就能让机器人习得这个新的六自由度操作技能，并将其永久存入技能库供后续任务复用。在仿真和真实场景的多项任务测试中，该系统成功完成了单凭原有技能库无法解决的任务变体，并展示了持续学习的能力。这一工作的意义在于，它首次在物理机器人上打通了"LLM 发现技能缺口→高效学习新技能→扩充技能库"的完整闭环，为迈向开放世界、终身学习的机器人系统提供了可行路径。
原文 arXiv:2309.14321;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2309.14321v2