Aha.
正在载入中英对照阅读…

arXiv:1506.08941 · 中英对照阅读

Language Understanding for Text-based Games using Deep Reinforcement Learning

Karthik Narasimhan*、Tejas D Kulkarni*、Regina Barzilay

中文速览

文字冒险游戏把世界状态藏在描述里,玩家必须读懂文字才能决定下一步,传统词袋方法却容易漏掉句子顺序和语义。研究者用深度强化学习,让 LSTM 阅读游戏描述、生成状态表示,再由神经网络给可选动作打分,并根据游戏奖励共同训练这两部分。实验中,这套方法在两个文字游戏里都比随机玩家和词袋基线表现更好,在一个奇幻游戏中完成了 96% 的任务;学到的文字表示还能迁移到其他游戏,帮助更快学会策略。它说明玩家可以直接从游戏反馈中学会理解文字和行动,不必预先标注每种状态,也为用自然语言操作复杂环境提供了思路。

摘要

In this paper, we consider the task of learning control policies for text-based games. In these games, all interactions in the virtual world are through text and the underlying state is not observed. The resulting language barrier makes such environments challenging for automatic game players. We employ a deep reinforcement learning framework to jointly learn state representations and action policies using game rewards as feedback. This framework enables us to map text descriptions into vector representations that capture the semantics of the game states. †† ∗Both authors contributed equally to this work. We evaluate our approach on two game worlds, comparing against baselines using bag-of-words and bag-of-bigrams for state representations. Our algorithm outperforms the baselines on both worlds demonstrating the importance of learning expressive representations. 11 1 Code is available at http://people.csail.mit.edu/karthikn/mud-play.

术语表

text-based games
文本游戏
control policy
控制策略
state representation
状态表示
deep reinforcement learning
深度强化学习
bag-of-words
词袋
bag-of-bigrams
二元词组袋
language grounding
语言落地
Markov Decision Process (MDP)
马尔可夫决策过程
action-value function
动作价值函数
Q-learning
Q-learning
deep recurrent neural network
深度循环神经网络
Long Short-Term Memory (LSTM)
Long Short-Term Memory(LSTM)
LSTM-DQN
LSTM-DQN
Multi-User Dungeon (MUD)
Multi-User Dungeon(MUD)
game state
游戏状态
textual description
文本描述
action-object pair
动作-对象对
stochastic transition function
随机转移函数
reward function
奖励函数
model-free technique
无模型方法
Bellman equation
贝尔曼方程
discount factor
折扣因子
state-action value function
状态-动作价值函数
representation layer
表示层
action selector
动作选择器
transfer learning
迁移学习
convergence of Q-values
Q值收敛