“No, to the Right” – Online Language Corrections for Robotic Manipulation via Shared AutonomyConference: Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction; March 13–16, 2023; Stockholm, SwedenProceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction (HRI ’23), March 13–16, 2023, Stockholm, SwedenDOI: 10.1145/3568162.3578623ISBN: 978-1-4503-9964-7/23/03CCS: Computing methodologies Cooperation and coordinationCCS: Computing methodologies Natural language processingCCS: Computing methodologies Learning from demonstrations
Yuchen Cui Note: Both authors contributed equally to this research. email: Affiliation: Stanford University , Stanford , CA , USA , Siddharth Karamcheti email: Affiliation: Stanford University , Stanford , CA , USA , Raj Palleti Affiliation: Stanford University , Stanford , CA , USA , Nidhya Shivakumar Affiliation: The Harker School , San Jose , CA , USA , Percy Liang Affiliation: Stanford University , Stanford , CA , USA and Dorsa Sadigh Affiliation: Stanford University , Stanford , CA , USA
Abstract
Systems for language-guided human-robot interaction must satisfy two key desiderata for broad adoption: adaptivity and learning efficiency. Unfortunately, existing instruction-following agents cannot adapt, lacking the ability to incorporate online natural language supervision, and even if they could, require hundreds of demonstrations to learn even simple policies. In this work, we address these problems by presenting Language-Informed Latent Actions with Corrections (LILAC), a framework for incorporating and adapting to natural language corrections – “to the right”, or “no, towards the book” – online, during execution. We explore rich manipulation domains within a shared autonomy paradigm. Instead of discrete turn-taking between a human and robot, LILAC splits agency between the human and robot: language is an input to a learned model that produces a meaningful, low-dimensional control space that the human can use to guide the robot. Each real-time correction refines the human’s control space, enabling precise, extended behaviors -- with the added benefit of requiring only a handful of demonstrations to learn. We evaluate our approach via a user study where users work with a Fran
原文 arXiv:2301.02555;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2301.02555v1