RT-1: Robotics Transformer for Real-World Control at Scale
Abstract
By transferring knowledge from large, diverse, task-agnostic datasets, modern machine learning models can solve specific downstream tasks either zero-shot or with small task-specific datasets to a high level of performance. While this capability has been demonstrated in other fields such as computer vision, natural language processing or speech recognition, it remains to be shown in robotics, where the generalization capabilities of the models are particularly critical due to the difficulty of collecting real-world robotic data. We argue that one of the keys to the success of such general robotic models lies with open-ended task-agnostic training, combined with high-capacity architectures that can absorb all of the diverse, robotic data. In this paper, we present a model class, dubbed Robotics Transformer, that exhibits promising scalable model properties. We verify our conclusions in a study of different model classes and their ability to generalize as a function of the data size, model size, and data diversity based on a large-scale data collection on real robots performing real-world tasks. The project’s website and videos can be found at robotics-transformer1.github.io
中文速览
机器人领域长期缺乏像GPT、CLIP那样能"举一反三"的通用大模型，根本原因在于真实机器人数据难以大规模收集，且现有模型要么任务范围窄、要么泛化能力弱。谷歌团队历时17个月、用13台真实机器人收集了逾13万条演示数据、覆盖700余项任务，并设计了一种名为RT-1（Robotics Transformer 1）的新型架构——它将摄像头图像、自然语言指令和机械臂动作压缩成紧凑的token序列，交由Transformer高效处理，使模型能在真实机器人上实时运行。实验结果表明，RT-1在已训练任务上的成功率高达97%，在新任务、新干扰物和新背景场景下的泛化能力分别比最强基线提升25%、36%和18%，还能融合仿真数据与不同机器人类型的数据，并支持多达50步的长程任务规划。这项工作首次在真实世界大规模验证了"数据越多、模型越大、任务越多样，机器人就越聪明"的规律，为构建通用机器人基础模型提供了有力证据。
原文 arXiv:2212.06817;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2212.06817v2