The Library as Dataset: Text Mining at Million-Book Scale
2,642 views · Published 18 October 2012 · 37:56 · Indexed 10 November 2026
Channel: Yale University · 2012 · Education
What do you do with a library? The large-scale digital collections scanned by Google and the Internet Archive have opened new ways to interact with books. The scale of digitization, however, also presents a challenge. We must find methods that are powerful enough to model the complexity of culture, but simple enough to scale to millions of books. In this talk I'll discuss one method, statistical topic modeling. I'll begin with an overview of the method. I will then demonstrate how to use such a model to measure changes over time and distinctions between sub-corpora. Finally, I will describe hypothesis tests that help us to distinguish consistent patterns from random variations. David Mimno is a postdoctoral researcher in the Computer Science department at Princeton University. He received his PhD from the University of Massachusetts, Amherst. Before graduate school, he served as Head Programmer at the Perseus Project, a digital library for cultural heritage materials, at Tufts University. He is supported by a CRA Computing Innovation fellowship.
More from this channel
-
1:37
Faith and Reconciliation
-
1:44
Faith and Human Rights
-
2:13
Faith and Economic Activity
-
2:53
"State of Faiths" in the World Today
-
2:02
Faiths and the Dynamics of Economic Development
-
2:29
Persons of Faith Who are Publicly Engaged
-
5:58
Tony Blair Sends Message To Class
-
25:37
Attridge and Kelsay on the Study of Religion