Solving Rubik’s Cube with a Robot Hand
OpenAI Ilge Akkaya, Marcin Andrychowicz††, Maciek Chociej††, Mateusz Litwin††, Bob McGrew††, Arthur Petron††, Alex Paino††, Matthias Plappert††, Glenn Powell††, Raphael Ribas††, Jonas Schneider††, Nikolas Tezak††, Jerry Tworek††, Peter Welinder††, Lilian Weng††, Qiming Yuan††, Wojciech Zaremba††, Lei Zhang††Authors are listed alphabetically. We include a detailed contribution statement at the end of this manuscript. Please cite as OpenAI et al., and use the following bibtex for citation: https://openai.com/bibtex/openai2019rubiks.bib
Abstract
We demonstrate that models trained only in simulation can be used to solve a manipulation problem of unprecedented complexity on a real robot. This is made possible by two key components: a novel algorithm, which we call automatic domain randomization (ADR) and a robot platform built for machine learning. ADR automatically generates a distribution over randomized environments of ever-increasing difficulty. Control policies and vision state estimators trained with ADR exhibit vastly improved sim2real transfer. For control policies, memory-augmented models trained on an ADR-generated distribution of environments show clear signs of emergent meta-learning at test time. The combination of ADR with our custom robot platform allows us to solve a Rubik’s cube with a humanoid robot hand, which involves both control and state estimation problems. Videos summarizing our results are available: https://openai.com/blog/solving-rubiks-cube/
中文速览
让机器人用灵巧手解魔方，是一个同时涉及精确控制和精确感知的超高难度任务，此前从未被实现过。研究团队提出了一种名为"自动域随机化"（Automatic Domain Randomization，ADR）的新算法，它能自动地、循序渐进地生成越来越难的仿真训练环境，让策略在不断扩大的随机化分布中反复磨炼，而无需人工设定随机化参数的范围。结合专门为机器学习定制的实体机器人平台，以及用仿真数据训练的视觉状态估计器，系统成功在真实的 Shadow 灵巧手上完成了对随机打乱的魔方的还原。更值得关注的是，搭载长短期记忆网络（LSTM）的控制策略在测试时表现出"涌现元学习"的迹象——它能在无需重新训练的情况下，通过内部状态自适应地应对现实环境的不确定性——这说明 ADR 加记忆网络的组合实际上是一种隐式的元学习框架，为解决仿真到现实（sim-to-real）迁移这一机器人领域核心难题提供了新思路。
原文 arXiv:1910.07113;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1910.07113v1