TextWorld: A Learning Environment for Text-based Games
Marc-Alexandre Côté Microsoft Research Ákos Kádár Tilburg University Xingdi Yuan Microsoft Research Ben Kybartas McGill University Tavian Barnes Microsoft Research Emery Fine Microsoft Research James Moore Microsoft Research Matthew Hausknecht Microsoft Research Ruo Yu Tao McGill University Layla El Asri Microsoft Research Mahmoud Adada Microsoft Research Wendy Tay Microsoft Research Adam Trischler Microsoft Research
Abstract
We introduce TextWorld, a sandbox learning environment for the training and evaluation of RL agents on text-based games. TextWorld is a Python library that handles interactive play-through of text games, as well as backend functions like state tracking and reward assignment. It comes with a curated list of games whose features and challenges we have analyzed. More significantly, it enables users to handcraft or automatically generate new games. Its generative mechanisms give precise control over the difficulty, scope, and language of constructed games, and can be used to relax challenges inherent to commercial text games like partial observability and sparse rewards. By generating sets of varied but similar games, TextWorld can also be used to study generalization and transfer learning. We cast text-based games in the Reinforcement Learning formalism, use our framework to develop a set of benchmark games, and evaluate several baseline agents on this set and the curated list.
中文速览
为了让强化学习（RL）智能体能在文本游戏这类复杂的自然语言交互环境中得到科学、可控的训练与评估，研究者推出了TextWorld这一沙盒学习框架。TextWorld不仅支持直接运行现有文本游戏，还能通过内置的逻辑引擎和上下文无关文法自动生成新游戏，并可精确调控游戏难度、地图大小、词汇丰富程度、奖励密度等参数，从而系统性地缓解部分可观测性和稀疏奖励等经典难题。基于这一框架，作者构建了一套基准游戏集，并对多个基线RL算法进行了评测，结果表明即便是经过简化的文本游戏场景对现有算法也颇具挑战性。TextWorld的意义在于：它为研究者提供了一个可重复、可扩展的"语言版训练场"，让课程学习、泛化能力和迁移学习等核心问题都能在文本游戏这一天然的语言理解与序列决策交叉地带上得到严格研究。
原文 arXiv:1806.11532;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1806.11532v2