Code as Policies: Language Model Programs for Embodied Control
Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, Andy Zeng Robotics at Google
Abstract
Large language models (LLMs) trained on code-completion have been shown to be capable of synthesizing simple Python programs from docstrings [1]. We find that these code-writing LLMs can be re-purposed to write robot policy code, given natural language commands. Specifically, policy code can express functions or feedback loops that process perception outputs (e.g., from object detectors [2, 3]) and parameterize control primitive APIs. When provided as input several example language commands (formatted as comments) followed by corresponding policy code (via few-shot prompting), LLMs can take in new commands and autonomously re-compose API calls to generate new policy code respectively. By chaining classic logic structures and referencing third-party libraries (e.g., NumPy, Shapely) to perform arithmetic, LLMs used in this way can write robot policies that (i) exhibit spatial-geometric reasoning, (ii) generalize to new instructions, and (iii) prescribe precise values (e.g., velocities) to ambiguous descriptions (“faster”) depending on context (i.e., behavioral commonsense). This paper presents Code as Policies: a robot-centric formulation of language model generated programs (LMPs) t
中文速览
用代码来表达机器人控制策略，是这项工作的核心思路：研究者发现，原本用于代码补全的大型语言模型（LLM，Large Language Model）可以被直接"借用"来为机器人生成控制代码，只需用自然语言指令作为注释输入即可。具体做法是，通过少样本提示（few-shot prompting）向模型展示若干"自然语言指令→对应Python代码"的例子，模型便能针对新指令自动组合感知API（如目标检测）和控制API，生成可直接在机器人上运行的策略代码，还能调用NumPy、Shapely等第三方库完成空间几何推理。实验结果表明，这套名为"Code as Policies"的方法不仅能让机器人理解"快一点""往左一点"这类模糊指令并给出合理的具体参数，还能通过层级式代码生成（hierarchical code-gen）将HumanEval代码生成基准的最优成绩提升至39.8%，并在多个真实机器人平台上成功完成反应式控制和基于路径点的操作任务。这项工作的意义在于，它绕开了昂贵的机器人数据采集与技能训练流程，让语言模型直接充当"策略编写者"，为更灵活、低成本的人机交互开辟了新路径。
原文 arXiv:2209.07753;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2209.07753v4