Rajesh Balamohan – Hadoop & Cloud Storage: Object Store Integration in Production

508 views · Published 31 July 2016 · 22:44 · Indexed 12 November 2026

Channel: Hasgeek TV · 2016 · Science & Technology

Watch on YouTube

Today’s typical Apache Hadoop deployments use HDFS for persistent, fault-tolerant storage of big data files. However, recent emerging architectural patterns increasingly rely on cloud object storage such as S3, Azure Blob Store, GCS, which are designed for cost-efficiency, scalability and geographic distribution. Hadoop supports pluggable file system implementations to enable integration with these systems for use cases such as off-site backup or even complex multi-step ETL, but applications may encounter unique challenges related to eventual consistency, performance and differences in semantics compared to HDFS.

More from this channel