Neural probabilistic motor primitives for humanoid control
Josh Merel, Leonard Hasenclever∗, Alexandre Galashov, Arun Ahuja, Vu Pham, Greg Wayne, Yee Whye Teh,、Nicolas Heess DeepMind London, UK {jsmerel,leonardh,agalashov,arahuja,vuph, Equal contribution.
Abstract
We focus on the problem of learning a single motor module that can flexibly express a range of behaviors for the control of high-dimensional physically simulated humanoids. To do this, we propose a motor architecture that has the general structure of an inverse model with a latent-variable bottleneck. We show that it is possible to train this model entirely offline to compress thousands of expert policies and learn a motor primitive embedding space. The trained neural probabilistic motor primitive system can perform one-shot imitation of whole-body humanoid behaviors, robustly mimicking unseen trajectories. Additionally, we demonstrate that it is also straightforward to train controllers to reuse the learned motor primitive space to solve tasks, and the resulting movements are relatively naturalistic. To support the training of our model, we compare two approaches for offline policy cloning, including an experience efficient method which we call linear feedback policy cloning. We encourage readers to view a supplementary video summarizing our results.
中文速览
为了让一个虚拟人形机器人能灵活地完成跑、跳、舞蹈等数千种动作，研究者提出了"神经概率运动原语"（Neural Probabilistic Motor Primitives，NPMP）系统：先用动作捕捉数据训练出2700多个单技能专家策略，再设计一个带隐变量瓶颈的逆模型网络，把这些专家的行为知识一次性"压缩"进一个共享的低维嵌入空间。训练完成后，该系统只需看一段从未见过的动作片段，就能立刻模仿出整体运动细节（单样本模仿），并且把学到的运动空间直接复用给下游任务控制器，生成的动作看起来相当自然流畅。为了高效完成这种大规模离线知识转移，研究者还提出了"线性反馈策略克隆"（Linear Feedback Policy Cloning，LFPC）方法，仅需少量参考轨迹就能媲美需要大量随机采样的传统行为克隆，大幅降低了计算成本。这项工作为构建能泛化、可复用、接近人类运动质感的通用人形体控制模块提供了一条切实可行的路径。
原文 arXiv:1811.11711;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1811.11711v2