Look Who it Was Random video

Rebecca Conley, Mark Lavin | Scraping Lyrics with Celery | PyData Carolinas 2016

708 views · Published 6 October 2016 · 25:33 · Indexed 10 November 2026

Channel: PyData · 2016 · Science & Technology

Watch on YouTube

This talk will demonstrate an example application of using Celery to extract all of the lyrics of the inspiring and influential Taylor Swift from the Internet. Using a light-hearted approach and practical Python examples, we aim to teach people the basics of using Celery for data extraction.

Celery is an open-source, Python-based, asynchronous task framework which is well-suited for extracting data from webpages, APIs, and text files. With its power comes a bit of a learning curve. This talk covers some of the first questions a new Celery user might have. It will also point out some common pitfalls for beginning users. 

00:00 Welcome!
1:20 About Celery
2:55 How Celery works
3:36 Message brokers
5:15 Choosing a broker
6:35 Storing results
9:30 Coding with Celery
15:30 Scrapping lyrics websites
16:46 Inspecting the DOM
17:13 Coding with BeautifulSoup
20:07 Getting the results and performance
22:40 Improvements: Better storage, Celery chains, groups, chords
24:15 Wrapping up: Celery is as popular as Taylor Swift for distributed tasks

S/o to https://github.com/mycaule for the video timestamps!

Want to help add timestamps to our YouTube videos to help with discoverability? Find out more here: https://github.com/numfocus/YouTubeVideoTimestamps

More from this channel