Look Who it Was Random video

Building Data Infrastructure at SocialCode

3,304 views · Published 23 September 2013 · 21:48 · Indexed 6 November 2026

Channel: InfoQ · 2013 · Science & Technology

Watch on YouTube

When we talk about scalability, we often talk about what happens when we have more product, but we don't always focus on social dynamics and how to scale a data platform in a more team oriented way. What happens when you're company grows from one office to three? How can you ensure that your ability to share data across teams grows with it? 

Such was the problem that Will Larson, VP of Platform Technology, faced at SocialCode, an analytics and optimization startup. This talks outlines their evolution, from monolithic Django/Python and MySQL shop, to a service oriented architecture with an underlying data pipeline for reliably aggregating content built on Kafka. Will discusses the solutions SocialCode discovered as they adapted to exponential growth.

Follow along with Will's slides: http://lethain.github.io/pres_data_pipeline/

More free Python developer resources: http://crcl.to/jvhzs
Profession Python training: http://crcl.to/qhsm9

Information on SF Python: http://sfpython.org

More from this channel