ViZDoom: A Doom-based AI Research Platform for Visual Reinforcement Learning
Michał Kempka, Marek Wydmuch, Grzegorz Runc, Jakub Toczek、Wojciech Jaśkowski Institute of Computing Science, Poznan University of Technology, Poznań, Poland
Abstract
The recent advances in deep neural networks have led to effective vision-based reinforcement learning methods that have been employed to obtain human-level controllers in Atari 2600 games from pixel data. Atari 2600 games, however, do not resemble real-world tasks since they involve non-realistic 2D environments and the third-person perspective. Here, we propose a novel test-bed platform for reinforcement learning research from raw visual information which employs the first-person perspective in a semi-realistic 3D world. The software, called ViZDoom, is based on the classical first-person shooter video game, Doom. It allows developing bots that play the game using the screen buffer. ViZDoom is lightweight, fast, and highly customizable via a convenient mechanism of user scenarios. In the experimental part, we test the environment by trying to learn bots for two scenarios: a basic move-and-shoot task and a more complex maze-navigation problem. Using convolutional deep neural networks with Q-learning and experience replay, for both scenarios, we were able to train competent bots, which exhibit human-like behaviors. The results confirm the utility of ViZDoom as an AI research platfor
中文速览
深度强化学习（deep reinforcement learning）在Atari游戏上已能达到人类水平，但那些游戏都是2D的、上帝视角的，和真实世界相差甚远。研究者们基于经典第一人称射击游戏《毁灭战士》（Doom）开发了一个名为ViZDoom的开源研究平台，让AI智能体只凭屏幕像素在三维第一人称视角环境中学习行动，并通过自定义场景机制支持灵活的实验设计。他们用卷积神经网络结合Q学习与经验回放（experience replay）在两个场景中训练智能体——一个是移动射击任务，另一个是更复杂的迷宫导航任务——结果均成功训练出了表现出类人行为的有效智能体。这项工作填补了视觉强化学习领域缺乏三维第一人称视角标准测试平台的空白，证明了在更接近真实世界的3D环境中进行纯视觉强化学习是完全可行的。
原文 arXiv:1605.02097;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1605.02097v2