Speaker(s):

New Avro serialization and deserialization in Beam SQL

17:15-18:05 CDT
Local room: 204

At Palo Alto Networks we heavily rely on Avro, using it as the primary storage format and use Beam Row as in memory. We de/serialize billions Avro records per second. One day we realized Avro Row conversion routines consume much of CPU time. Then the story begins ….