Latest News

Machine learning design patterns: between Beam and a hard place

In a recent book entitled Machine Learning Design Patterns, we captured best practices and solutions to recurring problems in machine learning. Many of these design patterns are best implemented using Beam. The obvious example is the Transform design pattern, which allows you to replicate arbitrary operations from the training graph in the serving graph while keeping both training and serving code efficient and maintainable. Indeed, the tf.transform package makes this easy.

By Beam Summit Team

Implementing Cloud Agnostic Machine Learning Workflows with Apache Beam on Kubernetes

The need for a highly efficient data processing workflow is fast becoming a necessity in every organization implementing and deploying Machine Learning models at scale. In most cases, ML teams leverage the managed service solutions already in place by the cloud infrastructure provider they choose. While this approach is good enough for most teams to get going, the long-term cost of keeping the platform running may be prohibitively higher over time.

By Beam Summit Team

Unified Streaming and Batch Pipelines at LinkedIn using Beam

Many use cases at LinkedIn require real-time processing and periodic backfilling of data. Running a single codebase for both needs is an emerging requirement. In this talk, we will share how we leverage Apache Beam to unify Samza stream and Spark batch processing. We will present the first unified production use case Standardization. By leveraging Beam on Spark for its backfilling, we reduced the backfilling time by 93% while only using 50% of resources.

By Beam Summit Team

Keynote: The adoption, current state and future of Apache Beam at Twitter

Lohit is part of Hadoop and Log Management team at Twitter. He has been concentrating on scaling Hadoop FileSystem, Hadoop Resource Manager, Log Ingestion and Processing pipelines at Twitter. During this talk he will explain the adoption, current state and future of Apache Beam at Twitter.

By Beam Summit Team

Keynote: Palo Alto Networks' massive-scale deployment of Beam

Talat is a Principal Software Engineer at Palo Alto Networks Cortex Data Lake Team, working on building a streaming data platform using Apache Beam and Dataflow to secure their customers. He will share how Beam was adopted and how they deploy thousands of pipelines.

By Beam Summit Team

Beam Playground: discover, learn and prototype with Apache Beam

This talk will be presenting the Apache Beam discovery and learning through Beam Playground, proof of concept and contributing paths to those who are looking to learn, use and develop Apache Beam.

By Beam Summit Team

Get your early bird tickets for Beam Summit 2022

Registration is now open for Beam Summit 2022. Online participation is free. Please get a free ticket in order to be able to access the sessions. Early bird pricing for in-person passes is $290 USD for 2-day pass and $350 USD for 3-day pass. If you would like to attend in person but cannot afford a ticket, please apply for a scholarship. Check out ticket options and prices.

By Beam Summit Team

Beam Summit 2022 CFP is open. We want to hear from you!

The “Call for Papers” for Beam Summit 2022 is open. We are looking for speakers that can provide technical insights, but also for people that can tell interesting stories on how and where “Apache Beam” is used. The following session types are suggested (if you have a good reason to deviate from the format please add this to your description): Session 25m: Live session of 25 minutes, ideal for a use-case.