Autoplay Next video from Max length

ABRIS: Avro Bridge for Apache Spark with Felipe Melo and Georgi Chochov (ABSA)

1,670 views · Published 11 October 2018 · 30:09 · Indexed 31 October 2026

Channel: Databricks · 2018 · Science & Technology

Avro has become the standard format for data serialization and data in motion. It provides rich and evolvable data structures and a compact, fast and language agnostic binary data format. Alongside the format itself, a number of instrumental technologies have been created to support schema management, such as Confluent’s Schema Registry. Currently, except for batch oriented jobs, the burden of integrating Avro with Spark is all on user’s shoulders.

Additionally, APIs for integrating with Schema Registry are still missing, making schema evolution in Spark applications that rely on Avro unnecessarily complicated. ABRiS solves those gaps by allowing applications using Spark structured APIs to seamlessly integrate with Avro. It provides conversions between Avro records and Spark Rows in a storage-agnostic manner and integrated with Schema Registry, all through a clean and simple API. In this presentation we cover how ABRiS works and demonstrate through simple examples how to use its features.

About: Databricks provides a unified data analytics platform, powered by Apache Spark™, that accelerates innovation by unifying data science, engineering and business.
Read more here: https://databricks.com/product/unified-data-analytics-platform

Connect with us:
Website: https://databricks.com
Facebook: https://www.facebook.com/databricksinc
Twitter: https://twitter.com/databricks
LinkedIn: https://www.linkedin.com/company/databricks
Instagram: https://www.instagram.com/databricksinc/ Databricks is proud to announce that Gartner has named us a Leader in both the 2021 Magic Quadrant for Cloud Database Management Systems and the 2021 Magic Quadrant for Data Science and Machine Learning Platforms. Download the reports here. https://databricks.com/databricks-named-leader-by-gartner

ABRIS: Avro Bridge for Apache Spark with Felipe Melo and Georgi Chochov (ABSA)

More from this channel