Ray: A Distributed Framework for Emerging AI Applications
Philipp Moritz111equal contribution, Robert Nishihara111equal contribution, Stephanie Wang, Alexey Tumanov, Richard Liaw, Eric Liang, Melih Elibol, Zongheng Yang, William Paul, Michael I. Jordan, Ion Stoica University of California, Berkeley
Abstract
The next generation of AI applications will continuously interact with the environment and learn from these interactions. These applications impose new and demanding systems requirements, both in terms of performance and flexibility. In this paper, we consider these requirements and present Ray—a distributed system to address them. Ray implements a unified interface that can express both task-parallel and actor-based computations, supported by a single dynamic execution engine. To meet the performance requirements, Ray employs a distributed scheduler and a distributed and fault-tolerant store to manage the system’s control state. In our experiments, we demonstrate scaling beyond 1.8 million tasks per second and better performance than existing specialized systems for several challenging reinforcement learning applications.
中文速览
强化学习（Reinforcement Learning, RL）应用需要同时支持策略训练、模拟仿真和在线决策三类紧密耦合的计算，而现有的大数据框架或深度学习框架各自只能覆盖其中一部分，拼接多个专用系统又因数据搬移和延迟代价过高而难以实用。为此，论文提出了 Ray——一个通用分布式计算框架，它将无状态的"任务并行"和有状态的"Actor"两种编程抽象统一到同一个动态任务图执行引擎之上，并通过分布式调度器和分布式容错元数据存储来突破单点瓶颈。实验结果显示，Ray 每秒可处理超过 180 万个任务，在多项挑战性 RL 基准上的性能也优于现有专用系统。这项工作的重要性在于，它为下一代持续与环境交互、边运行边学习的 AI 应用提供了一套兼顾高性能、高灵活性与容错能力的统一基础设施，大幅降低了构建大规模 RL 系统的工程门槛。
原文 arXiv:1712.05889;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1712.05889v2