Emergence of Grounded Compositional Language in Multi-Agent Populations
Igor Mordatch OpenAI San Francisco, California, USA、Pieter Abbeel UC Berkeley Berkeley, California, USA
Abstract
By capturing statistical patterns in large corpora, machine learning has enabled significant advances in natural language processing, including in machine translation, question answering, and sentiment analysis. However, for agents to intelligently interact with humans, simply capturing the statistical patterns is insufficient. In this paper we investigate if, and how, grounded compositional language can emerge as a means to achieve goals in multi-agent populations. Towards this end, we propose a multi-agent learning environment and learning methods that bring about emergence of a basic compositional language. This language is represented as streams of abstract discrete symbols uttered by agents over time, but nonetheless has a coherent structure that possesses a defined vocabulary and syntax. We also observe emergence of non-verbal communication such as pointing and guiding when language communication is unavailable.
中文速览
多智能体系统中，如何让AI自发"发明"一套有意义的语言来协调合作，一直是难题。研究者构建了一个二维物理仿真环境，让多个智能体（agent）在其中移动并通过抽象离散符号互相"说话"，用强化学习加端到端反向传播联合训练它们的行动和通信策略，整个系统无需任何人工标注或预设语言规则。实验发现，智能体自发涌现出一套具备词汇和语法结构的组合型语言（compositional language），会用不同符号分别指代地标、动作和其他智能体，甚至会优先说出"去"这个动词，让听者提前开始移动——这种顺序规律完全源于物理环境的约束。当语言通信被禁用时，智能体还能自发演化出"指向"和"引导"等非语言沟通行为。这项工作表明，扎根物理环境的多智能体强化学习是研究语言涌现机制的有效框架，也为理解人类语言组合性的起源提供了计算层面的实验证据。
原文 arXiv:1703.04908;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1703.04908v2