Design and development of a content-based music search engine

572 views · Published 6 September 2016 · 1:01:36 · Indexed 27 October 2026

Channel: Microsoft Research · 2016 · Science & Technology

Watch on YouTube

If you go to Amazon.com or the Apple Itunes store, your ability to search for  new music will largely be limited by the `query-by-metadata'   paradigm: search by song, artist or album name. However, when we talk  or write about music, we use a rich vocabulary of semantic concepts  to convey our listening experience. If we can model a relationship   between these concepts and the audio content, then we can produce a   more flexible music search engine based on a 'query-by-semantic-  description' paradigm.    In this talk, I will present a computer audition system that can both   annotate novel audio tracks with semantically meaningful words and   retrieve relevant tracks from a database of unlabeled audio content  given a text-base query. I consider the related tasks of content-  based audio annotation and retrieval as one supervised multi-class,  multi-label problem in which we model the joint probability of   acoustic features and words. For each word in a vocabulary, we use an  annotated corpus of songs to train a Gaussian mixture model (GMM)  over an audio feature space. We estimate the parameters of the model  using the weighted mixture hierarchies Expectation Maximization   algorithm. This algorithm is more scalable to large data sets and  produces better density estimates than standard parameter estimation  techniques. The quality of the music annotations produced by our  system is comparable with the performance of humans on the same task.   Our `query-by-semantic-description' system can retrieve appropriate  songs for a large number of musically relevant words. I also show  that our audition system is general by learning a model that can  annotate and retrieve sound effects.     Lastly, I will discuss three techniques for collecting the semantic  annotations of music that are needed to train such a computer  audition system. They include text-mining web documents, conducting  surveys, and deploying human computation games.

More from this channel