Harness the power of Spark SQL with the Data Source API by Stefano Baghino
476 views · Published 12 March 2018 · 19:16 · Indexed 8 November 2026
Channel: Devoxx · 2018 · Science & Technology
A deep dive into the Spark Data Source API, which allows arbitrary data sources to be exposed for consumption through Spark SQL itself. You will be able to go beyond natively supported formats like JSON or CSV and query your favorite non-relational database or even your own web services through the convenient and familiar interface of SQL, using Spark itself as a distributed query engine. A software engineer for the past eight years but a lifelong developer and learner, Stefano was lucky enough to test his skills on several platforms (Web, mobile and native) to address the need of several industries (power generation, energy automation, finance, advertising, transportation). He has contributed to the Apache Flink project and now works as a Data Engineer at eBay, enabling a team of Data Scientists to explore large data sets and extract valuable insights.
More from this channel
-
32:13
Writing Groovy AST Transformations: Getting Practical in an hour! by Baruch Sadogursky
-
25:58
TweetWallFX - Behind the Scenes by Sven Reimers
-
29:44
Second generation "workflow managers" for big data by Alex Van Boxel
-
29:13
Secrets are secrets. Please maintain them as secrets by Alex Soto
-
2:13:40
JavaScript build pipelines with Gulp.js by Stefan Baumgartner
-
2:04:09
Powerful Metaprogramming Techniques With Groovy by Jeff Brown
-
2:38:32
Distributed Machine Learning 101 using Apache Spark from a Browser by Xavier Tordoir/Andy Petrella
-
2:36:21
Caching 101: Caching on the JVM (and beyond) by Alex Snaps/Louis Jacomet