grounding language to autonomously- acquired skills via goal generation
Ahmed Akakzia Sorbonne Université、Cédric Colas∗ Inria \ANDPierre-Yves Oudeyer Inria、Mohamed Chetouani Sorbonne Université、Olivier Sigaud Sorbonne Université Equal contribution.
Abstract
We are interested in the autonomous acquisition of repertoires of skills. Language-conditioned reinforcement learning (lc-rl) approaches are great tools in this quest, as they allow to express abstract goals as sets of constraints on the states. However, most lc-rl agents are not autonomous and cannot learn without external instructions and feedback. Besides, their direct language condition cannot account for the goal-directed behavior of pre-verbal infants and strongly limits the expression of behavioral diversity for a given language input. To resolve these issues, we propose a new conceptual approach to language-conditioned rl: the Language-Goal-Behavior architecture (lgb). lgb decouples skill learning and language grounding via an intermediate semantic representation of the world. To showcase the properties of lgb, we present a specific implementation called decstr. decstr is an intrinsically motivated learning agent endowed with an innate semantic representation describing spatial relations between physical objects. In a first stage (g $\to$ b), it freely explores its environment and targets self-generated semantic configurations. In a second stage (l $\to$ g), it trains a lan
中文速览
让机器人像婴儿一样自主习得技能、再通过语言指令调用这些技能，一直是个难题——现有的语言条件强化学习方法要么依赖外部指令和奖励，要么把语言理解和技能学习捆绑在一起，导致同一条指令只能产生单一行为、失败后也无法换策略。为此，研究者提出了一种名为 LGB（Language-Goal-Behavior）的新架构，核心思路是在语言和行为之间插入一个语义中间层——由描述物体空间关系的谓词（如"above""close"）构成的语义配置空间——将技能学习与语言理解彻底解耦。具体实现称为 DECSTR：第一阶段，智能体完全自主探索，靠内在动机驱动自己设定和完成语义目标、积累技能库；第二阶段，用条件变分自编码器（C-VAE）学习把自然语言指令映射到满足该指令约束的语义配置，从而实现语言落地。实验表明，与端到端语言条件强化学习相比，LGB 能以多样化方式完成同一条指令、在一次尝试失败后自动切换策略，且语义中间表示比连续非语义表示更有助于语言理解——这为构建既能自主成长、又能听懂人话的机器人学习系统提供了一条新路径。
原文 arXiv:2006.07185;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2006.07185v3