ProAgent: Building Proactive Cooperative Agents with Large Language Models
Ceyao Zhang1,2\equalcontrib, Kaijie Yang3\equalcontrib, Siyi Hu4\equalcontrib, Zihao Wang2,5, Guanghe Li2, Yihang Sun2, Cheng Zhang2, Zhaowei Zhang2,5, Anji Liu2, Song-Chun Zhu5, Xiaojun Chang4, Junge Zhang3, Feng Yin1, Yitao Liang2, Yaodong Yang2 Work done when Ceyao Zhang visited Peking University.Corresponding author
Abstract
Building agents with adaptive behavior in cooperative tasks stands as a paramount goal in the realm of multi-agent systems. Current approaches to developing cooperative agents rely primarily on learning-based methods, whose policy generalization depends heavily on the diversity of teammates they interact with during the training phase. Such reliance, however, constrains the agents’ capacity for strategic adaptation when cooperating with unfamiliar teammates, which becomes a significant challenge in zero-shot coordination scenarios. To address this challenge, we propose ProAgent, a novel framework that harnesses large language models (LLMs) to create proactive agents capable of dynamically adapting their behavior to enhance cooperation with teammates. ProAgent can analyze the present state, and infer the intentions of teammates from observations. It then updates its beliefs in alignment with the teammates’ subsequent actual behaviors. Moreover, ProAgent exhibits a high degree of modularity and interpretability, making it easily integrated into various of coordination scenarios. Experimental evaluations conducted within the Overcooked-AI environment unveil the remarkable performance
中文速览
让AI智能体在协作任务中真正理解并配合陌生队友，是多智能体系统领域的核心难题——现有基于强化学习的方法严重依赖训练阶段与大量不同队友的交互，一旦遇到从未见过的合作伙伴便举步维艰。为此，研究者提出了ProAgent框架，利用大语言模型（Large Language Model, LLM）的推理能力，让智能体无需额外训练就能主动观察环境、推断队友意图，并通过"信念修正（Belief Correction）"机制持续校准对队友行为的预判，从而动态调整自身策略。在经典多智能体协作测试平台Overcooked-AI上的实验表明，ProAgent全面超越了五种基于自对弈和种群训练的方法，与模拟真实人类行为的代理模型合作时性能平均提升超过10%。这项工作的重要意义在于，它首次系统性地将LLM引入零样本协调（zero-shot coordination）场景，不仅无需昂贵的训练开销，还具备良好的可解释性和模块化扩展性，为构建能与人类自然协作的通用智能体提供了新思路。
原文 arXiv:2308.11339;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2308.11339v3