Extending Spark Machine Learning Pipelines Going beyond wordcount with Spark ML

153 views · Published 6 March 2018 · 11:15 · Indexed 20 November 2026

Channel: FOSDEM · 2018 · Science & Technology

Watch on YouTube

by Holden Karau

At: FOSDEM 2017

Apache Spark is one of the most popular new "big data" technologies, and nowhas a sci-kit-learn inspired pipeline API. This talk looks at how the pipelineAPI works as well as how to add your own custom algorithms to Apache Spark.

Apache Spark is one of the most popular new "big data" technologies, and nowhas a sci-kit-learn inspired pipeline API. This talk looks at how the pipelineAPI works as well as how to add your own custom algorithms to Apache Spark.The talk will be focused in Scala, but the same techniques can be used in Javaor with other JVM languages. Sadly extending the pipeline API can notcurrently be done in non-JVM languages, but the information on how to use thepipeline API will be useful to Python and R users as well.


Room: H.2213
Scheduled start: 2017-02-04 13:25:00

More from this channel