Multi-Agent Cooperation and the Emergence of (Natural) Language
Angeliki Lazaridou1 , Alexander Peysakhovich2, Marco Baroni2,3 1Google DeepMind, 2Facebook AI Research, 3University of Trento Work done while at Facebook AI Research.
Abstract
The current mainstream approach to train natural language systems is to expose them to large amounts of text. This passive learning is problematic if we are interested in developing interactive machines, such as conversational agents. We propose a framework for language learning that relies on multi-agent communication. We study this learning in the context of referential games. In these games, a sender and a receiver see a pair of images. The sender is told one of them is the target and is allowed to send a message from a fixed, arbitary vocabulary to the receiver. The receiver must rely on this message to identify the target. Thus, the agents develop their own language interactively out of the need to communicate. We show that two networks with simple configurations are able to learn to coordinate in the referential game. We further explore how to make changes to the game environment to cause the “word meanings” induced in the game to better reflect intuitive semantic properties of the images. In addition, we present a simple strategy for grounding the agents’ code into natural language. Both of these are necessary steps towards developing machines that are able to communicate wi
中文速览
让神经网络通过"交流游戏"自发习得语言，而非被动学习大量文本——这正是本文要探索的核心问题。研究者设计了一种"指代游戏"（referential game）：发送者（sender）和接收者（receiver）两个智能体同时看到一对图片，发送者须从固定词汇表中挑一个符号告知接收者哪张是目标图，接收者仅凭这个符号作出判断，双方都答对才算赢。实验结果显示，两个结构简单的神经网络在强化学习驱动下，能快速收敛到近乎完美的协调沟通；而通过调整游戏环境（例如使用更抽象的图像表征、改变干扰图的选取方式），智能体自发形成的"词义"会更贴近人类对物体类别的直觉判断。此外，研究还提出了一种将智能体内部编码与自然语言对接的简单方法，让人类能部分理解机器发展出来的"语言"。这项工作意义在于，它为构建真正能与人类互动交流的对话智能体提供了一条不依赖大规模标注数据、而依靠多智能体协作"逼出"语言涌现的新路径。
原文 arXiv:1612.07182;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1612.07182v2