Autoplay Next video from Max length

Building Streaming Recommendation Engines on Apache Spark with Rui Vieira (Red Hat)

2,789 views · Published 11 October 2018 · 27:01 · Indexed 30 October 2026

Channel: Databricks · 2018 · Science & Technology

Collaborative filtering is a well known method to implement recommendation engines. Although modern techniques, such as Alternating Least Squares (ALS), allow us to perform rating predictions with large amounts of observations, typically ALS is implemented as a distributed batch algorithm where retraining must be performed with the entirety of the data. However, when dealing with large amounts of data as a stream, batch retraining might be problematic. In this talk Rui will guide us in building a streaming ALS implementation using Apache Spark and based on Stochastic Gradient Descent, where training can be performed using observations as they arrive. The advantages of real-time streaming collaborative filtering will be discussed as well as the scenarios where batch ALS might be preferable.
Attachments area

About: Databricks provides a unified data analytics platform, powered by Apache Spark™, that accelerates innovation by unifying data science, engineering and business.
Read more here: https://databricks.com/product/unified-data-analytics-platform

Connect with us:
Website: https://databricks.com
Facebook: https://www.facebook.com/databricksinc
Twitter: https://twitter.com/databricks
LinkedIn: https://www.linkedin.com/company/databricks
Instagram: https://www.instagram.com/databricksinc/ Databricks is proud to announce that Gartner has named us a Leader in both the 2021 Magic Quadrant for Cloud Database Management Systems and the 2021 Magic Quadrant for Data Science and Machine Learning Platforms. Download the reports here. https://databricks.com/databricks-named-leader-by-gartner

Building Streaming Recommendation Engines on Apache Spark with Rui Vieira (Red Hat)

More from this channel