Josh Yudaken | Building domain specific databases using Python for prototypes during take off
475 views · Published 24 August 2016 · 33:19 · Indexed 12 November 2026
Channel: PyData · 2016 · Science & Technology
PyData SF 2016 Smyte is blocking fraudsters, spammers, scammers and harassers through analysis of web and mobile event data. There are hundreds of new database options cropping up, and while most of them are great they all make a range of different design choices which may/may not align with what you need. At Smyte we're solving infrastructure-heavy problems with a tiny engineering team. We've had to build custom databases in order to efficiently solve problems, but don't have the luxury of a ton of development time. In this talk I'll go through how we used Kafka & Python to implement a prototype Sliding HyperLogLog server. It held up for the six months we needed to, and some early decisions made it trivial to port the server to our new efficient C++ & RocksDB stack. 00:00 Welcome! 00:10 Help us add time stamps or captions to this video! See the description for details. Want to help add timestamps to our YouTube videos to help with discoverability? Find out more here: https://github.com/numfocus/YouTubeVideoTimestamps
More from this channel
-
31:23
Kostas Perifanos - Authorship Attribution Using Python
-
31:08
Prash Majmudar - Measuring the Digital Economy Using Big Data
-
36:26
Ian Ozsvald - The High Performance Python Landscape
-
37:48
Mike Muller - Shared Memory Parallelism with Python
-
35:01
Martin Goodson - Most Winning A/B Test Results are Illusory
-
30:19
Mark Basham - Python for High Throughput Science
-
1:13:42
David Cournapeau & Simon Jagoe - Narrowing the Gap Between Array Computing and Databases
-
31:38
Emlyn Clay - Phython, Pharmaceuticals & Discovery