Hierarchical Decision Making by Generating and Following Natural Language Instructions
Hengyuan Hu Facebook AI Research Denis Yarats New York University、Facebook AI Research Qucheng Gong Facebook AI Research Yuandong Tian Facebook AI Research Mike Lewis Facebook AI Research Equal Contribution.
Abstract
We explore using latent natural language instructions as an expressive and compositional representation of complex actions for hierarchical decision making. Rather than directly selecting micro-actions, our agent first generates a latent plan in natural language, which is then executed by a separate model. We introduce a challenging real-time strategy game environment in which the actions of a large number of units must be coordinated across long time scales. We gather a dataset of 76 thousand pairs of instructions and executions from human play, and train instructor and executor models. Experiments show that models using natural language as a latent variable significantly outperform models that directly imitate human actions. The compositional structure of language proves crucial to its effectiveness for action representation. We also release our code, models and data111A demo is available at www.minirts.net222Our code is open-sourced at www.github.com/facebookresearch/minirts.
中文速览
用自然语言指令作为"中间计划"来玩即时战略游戏，能不能让AI打得更好？研究者构建了一个简化版即时战略游戏环境，收集了7.6万条人类玩家的"指挥官发指令、执行者操作"配对数据，训练出一个两层架构的智能体：高层模型先用自然语言生成战术指令，低层模型再把指令转化为具体的单位操作。实验结果表明，以自然语言作为中间隐变量的智能体，胜率显著优于直接模仿人类操作的端到端模型，而且语言的组合性（compositionality）是关键——能把词义组合起来的模型比把每条指令当成孤立符号的模型泛化能力更强。这项工作说明，人类语言不仅是沟通工具，还可以作为AI分层决策的天然"行动表示"，为解决复杂长时序任务提供了一条兼具可解释性和可扩展性的新路径。
原文 arXiv:1906.00744;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1906.00744v5