The Ray Beam Runner Project: A Vision for Unified Batch, Streaming, and ML

The Ray Beam Runner Project is an initiative to create a new Pythonic Beam Runner based on the Ray distributed compute framework. The project was conceived based on strong community interest in integrating Ray with Beam, and a prototype quickly proved its viability. We will discuss the current state of this initiative, and our long-term vision to provide a unified authoring and execution environment for mixed-purpose batch, streaming, and ML pipelines.