Master Class: Lecture 3

SpeakerMartin Wainright
AffiliationUC Berkeley
DateWednesday, 27 Feb 2013
Time13:00 - 14:00
LocationRoberts G06 Sir Ambrose Fleming LT
Event seriesMaster Class: Martin Wainwright (25 Feb - 1 Mar 2013)

High-dimensional statistics deals with problems in which the number of samples n is of the same order as, or substantially smaller than the ambient dimension p of the data. The study of such "large p, small n" problems dates back to work of Kolmogorov, and has been the subject of intensive study over the past several decades. Of course, these problems are ill-posed without further restrictions, and so a large body of research has focused on models that endowed with some form of low-dimensional structure. Examples include vectors that are sparse (with relatively few non-zero entries), matrices that are sparse and/or low-rank, and regression functions that are defined on manifolds.

In these lectures, we survey certain aspects of this rapidly evolving field, beginning with sparse vector estimation and its applications to graphical model selection, before moving onto more general high-dimensional M-estimators, and concluding with a look at high-dimensional non-parametrics.

Lecture 3: From parametric to non-parametric: The ``curse-of-dimensionality'' is even more severe for non-parametric models. In this lecture, we discuss various classes of non-parametric regression models with low-dimensional structure, and then explore in detail the class of sparse additive models over reproducing kernel Hilbert spaces. We present a simple estimator based on solving a second-order cone program (SOCP), and discuss some of the challenges involved in proving rates for non-parametric methods under high-dimensional scaling. Lastly, we show that the rates achieved by the SOCP method are minimax-optimal.

Slides: Wainwright_Lecture3.pdf


iCalendar csml_id_111.ics