Autoplay Next video from Max length

Apache Spark for RDBMS Practitioners: How I Learned to Stop Worrying and Love to Scale- Luca Canali

1,557 views · Published 10 October 2018 · 31:45 · Indexed 31 October 2026

Channel: Databricks · 2018 · Science & Technology

Fuse graph, document and relational data from transactional and analytic data sources, into a property graph "bird's eye view". The property graph data model is Chen's "entity relationship" model, without clutter. Use "ASCII Art" visual property graph schemas to define "graph data lifts", mapping from data lake, RDBMS, RDF or graph data cloud services into Spark. Graphs in Spark draw on multiple data sources. Leverage the Cypher query language to combine, split, and project graphs in Spark memory. Graph data is "woven" in Spark without altering or copying the original source. The results of graph workloads can be written back into HDFS or other file systems. Graphs can be read from, stored and merged into a Neo4j transactional database. And tabular datasets can be extracted from graphs. Data scientists and engineers load, wrangle and analyze mixed model data through Morpheus transformations. Enterprises use graphs to catalogue their disparate data assets and processes. They store graph datasets in the data lake. In a world of concern about data protection, see how graph data lifts allow tailored, canonical data views to be realized, in Spark, without remodeling and moving data. Morpheus combines SparkSQL and Cypher queries, and table/graph functions.Choose the right language for the job: eliminate cumbersome multi-joins for connected-data traversals by using super-concise Cypher patterns for sub-graph detection and graph projection; use the power of table projection, grouping, aggregation in SparkSQL, all in one application. Feel free to "dismantle your graph": expose your graph nodes or relationships as dataframes, or as Hive tables. Key Takeaways Graph technology meets Big Data and Spark Analytics Property graphs: the superset data model Graph, relational and document data, interwoven Lift, split, combine, and create new graphs, from any data source Get your data fit to exploit graph compute, without losing any of your existing tools undefined undefined undefined undefined undefined

About: Databricks provides a unified data analytics platform, powered by Apache Spark™, that accelerates innovation by unifying data science, engineering and business.
Read more here: https://databricks.com/product/unified-data-analytics-platform

Connect with us:
Website: https://databricks.com
Facebook: https://www.facebook.com/databricksinc
Twitter: https://twitter.com/databricks
LinkedIn: https://www.linkedin.com/company/databricks
Instagram: https://www.instagram.com/databricksinc/ Databricks is proud to announce that Gartner has named us a Leader in both the 2021 Magic Quadrant for Cloud Database Management Systems and the 2021 Magic Quadrant for Data Science and Machine Learning Platforms. Download the reports here. https://databricks.com/databricks-named-leader-by-gartner

Apache Spark for RDBMS Practitioners: How I Learned to Stop Worrying and Love to Scale- Luca Canali

More from this channel