Aha.
正在载入中英对照阅读…

arXiv:2004.04136 · 中英对照阅读

CURL: Contrastive Unsupervised Representations for Reinforcement Learning

Aravind Srinivas∗、Michael Laskin∗、Pieter Abbeel

中文速览

像素画面信息量大,导致强化学习要反复试很多次才能学会控制,效率远不如直接读取状态数据。CURL把对比学习(contrastive learning)和强化学习结合起来,让智能体在练习控制的同时,学会辨认画面中对任务有用的信息,再用这些特征做决策。实验显示,它在控制任务和电子游戏上都比此前的像素方法更省交互数据;在一组连续控制任务中,表现已接近使用状态数据的方法。这个结果说明,经过合适的表示学习,智能体有望只看图像也能高效学习,从而减少模拟实验和真实机器人的训练成本。

摘要

We present CURL: Contrastive Unsupervised Representations for Reinforcement Learning. CURL extracts high-level features from raw pixels using contrastive learning and performs off-policy control on top of the extracted features. CURL outperforms prior pixel-based methods, both model-based and model-free, on complex tasks in the DeepMind Control Suite and Atari Games showing 1.9x and 1.2x performance gains at the 100K environment and interaction steps benchmarks respectively. On the DeepMind Control Suite, CURL is the first image-based algorithm to nearly match the sample-efficiency of methods that use state-based features. Our code is open-sourced and available at https://www.github.com/MishaLaskin/curl.

术语表

CURL
对比式无监督强化学习表征
Contrastive Learning
对比学习
Unsupervised Representation Learning
无监督表征学习
Reinforcement Learning (RL)
强化学习
Sample Efficiency
样本效率
Pixel-Based Methods
基于像素的方法
High-Dimensional Observations
高维观测
Raw Pixels
原始像素
Off-Policy Control
离策略控制
Model-Based Reinforcement Learning
基于模型的强化学习
Model-Free Reinforcement Learning
无模型强化学习
Auxiliary Tasks
辅助任务
Self-Supervised Learning
自监督学习
Semantic Representation
语义表征
World Model
世界模型
Predictive Model
预测模型
Contrastive Objective
对比目标
Data Augmentation
数据增强
Temporal Frame Stack
时序帧堆叠
Soft Actor-Critic (SAC)
软演员-评论家算法
Dreamer
Dreamer
DeepMind Control Suite (DMControl)
DeepMind 控制套件
Atari Games
Atari 游戏
Rainbow DQN
Rainbow DQN
Efficient Rainbow
Efficient Rainbow
SimPLe
SimPLe
Contrastive Predictive Coding (CPC)
对比预测编码
State-Based Features
基于状态的特征