Embodied BERT: A Transformer Model for Embodied, Language-guided Visual Task Completion
Alessandro Suglia1、Qiaozi Gao2、Jesse Thomason2,3 \ANDGovind Thattai2、Gaurav S. Sukhatme2,3 \AND1Heriot-Watt University; 2Amazon Alexa AI; 3University of Southern California Work completed via internship with Amazon Alexa AI.
Abstract
Language-guided robots performing home and office tasks must navigate in and interact with the world. Grounding language instructions against visual observations and actions to take in an environment is an open challenge. We present Embodied BERT (EmBERT), a transformer-based model which can attend to high-dimensional, multi-modal inputs across long temporal horizons for language-conditioned task completion.Additionally, we bridge the gap between successful object-centric navigation models used for non-interactive agents and the language-guided visual task completion benchmark, ALFRED, by introducing object navigation targets for EmBERT training. EmBERT achieves competitive performance on the ALFRED benchmark, and is the first model to use a full, pretrained BERT stack while handling the long-horizon, dense, multi-modal histories of ALFRED. Model code is available at the following link: https://github.com/amazon-research/embert
中文速览
为了让机器人在家居或办公场景中执行自然语言指令，研究者需要解决如何将语言、视觉观察与具体动作三者有效对齐这一难题。本文提出了 Embodied BERT（EmBERT），一个基于 Transformer 的多模态模型：它一方面借用预训练的 BERT 堆栈同时编码语言指令与视觉目标检测信息，另一方面通过"分段循环动作解码器"将长达平均 50 步的操作轨迹切分为短段逐段处理，从而绕开 Transformer 处理长序列的计算瓶颈；更关键的是，作者为导航动作引入了"以物体为中心的导航目标"监督信号，让模型在移动过程中也能预测目标物体的空间位置，而非只在操作动作时才关注物体。在 ALFRED 基准测试上，EmBERT 取得了有竞争力的成绩，成为首个在该任务上完整使用预训练 BERT 并处理长时序密集多模态历史的模型，证明了将对象导航思路引入交互式具身智能（Embodied AI）任务的可行性与价值。
原文 arXiv:2108.04927;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2108.04927v2