GPT Models Meet Robotic Applications: Co-Speech Gesturing Chat System
Naoki Wake1, Atsushi Kanehira1, Kazuhiro Sasabuchi1, Jun Takamatsu1, and Katsushi Ikeuchi1 1Applied Robotics Research, Microsoft, Redmond, WA 98052, USA
Abstract
This technical paper introduces a chatting robot system that utilizes recent advancements in large-scale language models (LLMs) such as GPT-3 and ChatGPT (Fig.1). The system is integrated with a co-speech gesture generation system, which selects appropriate gestures based on the conceptual meaning of speech. Our motivation is to explore ways of utilizing the recent progress in LLMs for practical robotic applications, which benefits the development of both chatbots and LLMs. Specifically, it enables the development of highly responsive chatbot systems by leveraging LLMs and adds visual effects to the user interface of LLMs as an additional value. The source code for the system is available on GitHub for our in-house robot and GitHub for Toyota HSR.
中文速览
随着GPT-3、ChatGPT等大规模语言模型（LLMs）的兴起，如何把它们真正用在机器人上、让人与机器人的对话既自然又生动，成为一个值得探索的实际问题。研究团队为此搭建了一套聊天机器人系统，以GPT-3或ChatGPT作为对话引擎，同时接入一套"伴语手势"生成模块——系统会分析机器人说话的语义概念，从预设手势库中自动挑选并播放匹配的肢体动作，让机器人边说边做手势。该系统已在自研小型机器人MSRAbot和丰田HSR两款平台上落地验证，手势动作以拉班舞谱（Labanotation）这一通用中间格式存储，理论上可方便地移植到任意机器人，相关代码也已完全开源。这项工作的意义在于：一方面展示了用简单提示词工程（prompt engineering）就能快速构建高响应度聊天机器人的可行路径，另一方面也为探索"视觉化身体语言能否改善以文字为主的LLM交互体验"这一开放问题提供了实验平台。
原文 arXiv:2306.01741;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2306.01741v1