Autoplay Next video from Max length

Andrew Stretton - Twisted up in a Distributed Tornado

1,187 views · Published 6 May 2017 · 1:09:43 · Indexed 31 October 2026

Channel: PyData · 2017 · Science & Technology

Twisted up in a Distributed Tornado - a beginners guide to async frameworks in python

Filmed at PyData 2017

Description
By coming to this tutorial you will learn:

What event-driven Python Frameworks are and why they are useful
How to overcome any fears you may have about event-driven Python programming
Enough examples to be able to consider projects like Dask/Distributed and Tornado for your next project
Abstract
At some point in a career in data science, software development or data engineering you will be looking to develop a piece of code that runs alongside another piece of code. For example, you might want to:

Use all of your processors when running a script
Consume social media data as it is created
Create a super simple task scheduler without an ugly while True loop
Understand how Jupyter notebooks work
Write to a database and move on to the next task without waiting for confirmation
Call a function and throw an exception if it takes too long
There are numerous ways of achieving such concurrency. If creating a project to run on one or a few machines which needs good exception handling then I find tools like Twisted, Tornado and Dask/Distributed ideal for the above tasks.

As a python developer I have always tended to work with Celery and felt intimidated by the different flow control when using async tools like Twisted and Tornado. In this tutorial I will walk through the examples above and talk about how I went from Async novice to Dask/Distributed contributor in 6 months. The tutorial will concentrate mainly on practical use cases. Along the way we will encounter some bumps in the road which will hopefully aid attendees' understanding of the basic dos and don'ts around these tools.

By coming to this tutorial you will learn:

www.pydata.org

PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limited to) Python, Julia, and R.

We aim to be an accessible, community-driven conference, with novice to advanced level presentations. PyData tutorials and talks bring attendees the latest project features along with cutting-edge use cases. 00:00 Welcome!
00:10 Help us add time stamps or captions to this video! See the description for details.

Want to help add timestamps to our YouTube videos to help with discoverability? Find out more here: https://github.com/numfocus/YouTubeVideoTimestamps

Andrew Stretton - Twisted up in a Distributed Tornado

More from this channel