Autoplay Next video from Max length

Sparser - Faster Parsing of Unstructured Data Formats in Apache Spark (Firas Abuzaid)

2,140 views · Published 24 September 2018 · 30:24 · Indexed 28 October 2026

Channel: Databricks · 2018 · Science & Technology

Firas Abuzaid is a 3rd-year PhD student in the Stanford InfoLab, advised Profs.
In this talk, we present Sparser, a new parsing library in Spark for JSON, CSV, and Avro files. By aggressively filtering records before parsing them, Sparser achieves up to 9x end-to-end runtime improvement on several real-world Spark SQL workloads. Using Spark’s Data Source API, Sparser extracts the filtering expressions specified by a Spark SQL query; these expressions are then compiled into fast, SIMD-accelerated “pre-filters” which can discard data at an order of magnitude faster than the JSON and CSV parsers currently available in Spark.

To learn more: https://databricks.com/product/getting-started-guide
About: Databricks provides a unified data analytics platform, powered by Apache Spark™, that accelerates innovation by unifying data science, engineering and business.
Read more here: https://databricks.com/product/unified-data-analytics-platform

Connect with us:
Website: https://databricks.com
Facebook: https://www.facebook.com/databricksinc
Twitter: https://twitter.com/databricks
LinkedIn: https://www.linkedin.com/company/databricks
Instagram: https://www.instagram.com/databricksinc/ Databricks is proud to announce that Gartner has named us a Leader in both the 2021 Magic Quadrant for Cloud Database Management Systems and the 2021 Magic Quadrant for Data Science and Machine Learning Platforms. Download the reports here. https://databricks.com/databricks-named-leader-by-gartner

Sparser - Faster Parsing of Unstructured Data Formats in Apache Spark (Firas Abuzaid)

More from this channel