Supervised Dimension Reduction

378 views · Published 12 August 2016 · 40:16 · Indexed 1 November 2026

Channel: Microsoft Research · 2016 · Science & Technology

Watch on YouTube

We look at the problem of supervised dimension reduction (SDR) from three perspectives. The SDR problem is given observations, x-y pairs, infer a subspace of the input data without losing any predictive accuracy. We first study it as a geometric problem, show that the gradient of the regression or classification function is a central quantity, and in the case where the inputs are concentrated on manifold the covergence of estimators will be a function of the dimension of the manifold and not the ambient space. We then examine the same problem from a probabilistic modeling perspective and give a Bayesian solution. In this setting factor models and the Grassmann manifold are central. We close with an algorithmic perspective where we show that randomized algorithms can be used for SDR on massive data. We discuss the statistical implications of these randomized algorithms, they are a form or regularization.

More from this channel