Autoplay Next video from Max length

Tariq Rashid - Dimension Reduction and Extracting Topics - A Gentle Introduction

1,468 views · Published 15 May 2017 · 37:51 · Indexed 24 October 2026

Channel: PyData · 2017 · Science & Technology

Filmed at PyData 2017

Description
Text mining has many powerful methods for unlocking insights into the messy, ambiguous, but interesting text created by people.

Singular value decomposition (SVD) is a useful method for reducing the many dimensions of text data, and distill out key themes in that text - called topic modelling or latent semantic analysis.

This talk for beginners will gently explain SVD and how to use it.

Abstract
Text mining and natural language processing are hugely powerful fields that can unlock insights into the vast amounts of human knowledge, creativity and drivel (!) for automated computing. Examples include the fun of highlighting trends in internet chatter through to more serious analysis of finding patterns and links in leaked data sets of public interest.

One key tool is to reduce the many dimensions of text data, and distill out the key themes in that text. People call this topic modelling, latent semantic analysis, and a few other names too. The powerful method at the heart of this is called singular value decomposition (SVD).

This talk will gently introduce singular valued decomposition (SVD), explaining the mathematics in an accessible manner, and demonstrate how it can be used, using the Chilcot Iraq Report as an example dataset.

Example code, notebooks and data sets are public on GitHub, and there is a blog for more discussion of this, and other text mining ideas http://makeyourowntextminingtoolkit.blogspot.co.uk

www.pydata.org

PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limited to) Python, Julia, and R.

We aim to be an accessible, community-driven conference, with novice to advanced level presentations. PyData tutorials and talks bring attendees the latest project features along with cutting-edge use cases. 00:00 Welcome!
00:10 Help us add time stamps or captions to this video! See the description for details.

Want to help add timestamps to our YouTube videos to help with discoverability? Find out more here: https://github.com/numfocus/YouTubeVideoTimestamps

Tariq Rashid - Dimension Reduction and Extracting Topics - A Gentle Introduction

More from this channel