Sadayuki Furuhashi | Fighting Against Chaotically Separated Values with Embulk
1,216 views · Published 24 August 2016 · 42:10 · Indexed 20 October 2026
Channel: PyData · 2016 · Science & Technology
PyData SF 2016 Python is a great tool for performing data analysis, but often time the hardest part is getting access to your data that’s located in a variety of business systems - files, database, and SaaS applications. Productionizing this process is even harder: scripts frequently fail and require precious to to fix and re-test. In this talk, I will review some open source tools I authored and show you how In this talk we will cover: - How we created a data collection tool that can read any chaotically formatted files called "CSV" by guessing its structure automatically - Explore the plugin-based-architecture that makes it easy to load data from external sources and publish to production systems. From files to business systems such as Salesforce & Mixpanel. - Review current plugins (over 100 released by the OSS community) and use cases - Explain how distributed execution enhances stability and scalability 00:00 Welcome! 00:10 Help us add time stamps or captions to this video! See the description for details. Want to help add timestamps to our YouTube videos to help with discoverability? Find out more here: https://github.com/numfocus/YouTubeVideoTimestamps
More from this channel
-
31:23
Kostas Perifanos - Authorship Attribution Using Python
-
31:08
Prash Majmudar - Measuring the Digital Economy Using Big Data
-
36:26
Ian Ozsvald - The High Performance Python Landscape
-
37:48
Mike Muller - Shared Memory Parallelism with Python
-
35:01
Martin Goodson - Most Winning A/B Test Results are Illusory
-
30:19
Mark Basham - Python for High Throughput Science
-
1:13:42
David Cournapeau & Simon Jagoe - Narrowing the Gap Between Array Computing and Databases
-
31:38
Emlyn Clay - Phython, Pharmaceuticals & Discovery