Seminar: Efficient score estimation with infinite-dimensional exponential families

SpeakerDougal Sutherland
AffiliationUCL, Gatsby
DateFriday, 15 Dec 2017
Time13:00 - 14:00
LocationRoberts Building G08 Sir David Davies LT
Event seriesDeepMind CSML Seminar Series
Description

The infinite-dimensional exponential family is a rich generalization of the standard exponential family, going beyond finite sufficient statistic functions to allow for very complex models. In particular, we study the kernel exponential family, where the natural parameter lies in a reproducing kernel Hilbert space. Computing the normalization constant in this class of models is difficult, but efficient estimation is possible via score matching. This approach, however, has cubic computational complexity in both the number of sampled points and their dimension. We thus propose estimation with a low-rank, Nyström-like approximation. The new solution retains essentially the same convergence rate of the full-rank solution, with substantially less computational effort and storage. We demonstrate the applicability of the method both to density estimation and to approximating Hamiltonian Monte Carlo when gradients are available.

iCalendar csml_id_330.ics