Automatic Discovery of the Statistical Types of Variables in a Dataset

606 views · Published 19 May 2017 · 22:10 · Indexed 27 October 2026

Channel: Microsoft Research · 2017 · Science & Technology

Watch on YouTube

A common practice in statistics and machine learning is to assume that the statistical data types (e.g., ordinal, categorical or real-valued) of variables, and usually also the likelihood model, is known. However, as the availability of real-world data increases, this assumption becomes too restrictive. Data are often heterogeneous, complex, and improperly or incompletely documented. Surprisingly, despite their practical importance, there is still a lack of tools to automatically discover the statistical types of, as well as appropriate likelihood (noise) models for, the variables in a dataset. In this work, we fill this gap by proposing a Bayesian method, which accurately discovers the statistical data types in both synthetic and real data.

See more on this video at https://www.microsoft.com/en-us/research/video/automatic-discovery-statistical-types-variables-dataset/

More from this channel