MinAtar: An Atari-Inspired Testbed for Thorough and Reproducible Reinforcement Learning Experiments
Kenny Young Department of Computing Science University of Alberta Edmonton, AB, Canada Tian Tian Department of Computing Science University of Alberta Edmonton, AB, Canada
Abstract
The Arcade Learning Environment (ALE) is a popular platform for evaluating reinforcement learning agents. Much of the appeal comes from the fact that Atari games demonstrate aspects of competency we expect from an intelligent agent and are not biased toward any particular solution approach. The challenge of the ALE includes (1) the representation learning problem of extracting pertinent information from raw pixels, and (2) the behavioural learning problem of leveraging complex, delayed associations between actions and rewards. Often, the research questions we are interested in pertain more to the latter, but the representation learning problem adds significant computational expense. We introduce MinAtar, short for miniature Atari, a new set of environments that capture the general mechanics of specific Atari games while simplifying the representational complexity to focus more on the behavioural challenges. MinAtar consists of analogues of five Atari games: Seaquest, Breakout, Asterix, Freeway and Space Invaders. Each MinAtar environment provides the agent with a $10\times 10\times n$ binary state representation. Each game plays out on a $10\times 10$ grid with $n$ channels corresp
中文速览
强化学习研究者普遍用雅达利游戏（Atari）作为测试平台，但原始像素输入带来的表征学习（representation learning）开销极大，导致实验往往只能跑很少的随机种子，结论可靠性存疑。为此，作者推出了 MinAtar——五款迷你版雅达利游戏，把游戏画面压缩为 10×10 的语义通道网格，保留核心玩法机制的同时大幅降低表征复杂度，让研究者能把计算资源集中在行为学习（behavioural learning）本身。实验表明，即便在这套简化环境中，DQN 和在线 Actor-Critic 等算法仍面临有趣的探索与策略挑战，而节省下来的算力则被用于系统性地扫描超参数、增加实验重复次数。MinAtar 填补了"玩具任务"与完整 ALE 之间的空白，为更可复现、更严谨的强化学习研究提供了一个轻量级的中间测试床。
原文 arXiv:1903.03176;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1903.03176v2