arXiv:1506.08941 · 中英对照阅读
Language Understanding for Text-based Games using Deep Reinforcement Learning
中文速览
文字冒险游戏把世界状态藏在描述里,玩家必须读懂文字才能决定下一步,传统词袋方法却容易漏掉句子顺序和语义。研究者用深度强化学习,让 LSTM 阅读游戏描述、生成状态表示,再由神经网络给可选动作打分,并根据游戏奖励共同训练这两部分。实验中,这套方法在两个文字游戏里都比随机玩家和词袋基线表现更好,在一个奇幻游戏中完成了 96% 的任务;学到的文字表示还能迁移到其他游戏,帮助更快学会策略。它说明玩家可以直接从游戏反馈中学会理解文字和行动,不必预先标注每种状态,也为用自然语言操作复杂环境提供了思路。
摘要
In this paper, we consider the task of learning control policies for text-based games. In these games, all interactions in the virtual world are through text and the underlying state is not observed. The resulting language barrier makes such environments challenging for automatic game players. We employ a deep reinforcement learning framework to jointly learn state representations and action policies using game rewards as feedback. This framework enables us to map text descriptions into vector representations that capture the semantics of the game states. †† ∗Both authors contributed equally to this work. We evaluate our approach on two game worlds, comparing against baselines using bag-of-words and bag-of-bigrams for state representations. Our algorithm outperforms the baselines on both worlds demonstrating the importance of learning expressive representations. 11 1 Code is available at http://people.csail.mit.edu/karthikn/mud-play.
术语表
- text-based games
- 文本游戏
- control policy
- 控制策略
- state representation
- 状态表示
- deep reinforcement learning
- 深度强化学习
- bag-of-words
- 词袋
- bag-of-bigrams
- 二元词组袋
- language grounding
- 语言落地
- Markov Decision Process (MDP)
- 马尔可夫决策过程
- action-value function
- 动作价值函数
- Q-learning
- Q-learning
- deep recurrent neural network
- 深度循环神经网络
- Long Short-Term Memory (LSTM)
- Long Short-Term Memory(LSTM)
- LSTM-DQN
- LSTM-DQN
- Multi-User Dungeon (MUD)
- Multi-User Dungeon(MUD)
- game state
- 游戏状态
- textual description
- 文本描述
- action-object pair
- 动作-对象对
- stochastic transition function
- 随机转移函数
- reward function
- 奖励函数
- model-free technique
- 无模型方法
- Bellman equation
- 贝尔曼方程
- discount factor
- 折扣因子
- state-action value function
- 状态-动作价值函数
- representation layer
- 表示层
- action selector
- 动作选择器
- transfer learning
- 迁移学习
- convergence of Q-values
- Q值收敛