Tanmay K. Mohapatra - Crunching Big Data with Julia

349 views · Published 30 October 2015 · 19:37 · Indexed 14 November 2026

Channel: Hasgeek TV · 2015 · Science & Technology

Watch on YouTube

This talk will use Elly.jl to demonstrate a big data workflow in Julia. Elly is a Hadoop HDFS and Yarn client. It is a pure Julia implementation with no dependencies on libhdfs. It provides:

A familiar Julia ClusterManager interface, making it possible to use the familiar Julia parallel constructs on a Yarn cluster: addprocs, @parallel, spawn, pmap, etc.
Lower level APIs to write native Yarn applications.
A familiar Julia IO API for accessing HDFS files.
We shall use Elly and a few associated Julia packages to process a few example datasets.

https://juliacon.talkfunnel.com/2015/16-crunching-big-data-with-julia

More from this channel