Seminar: Variational Hierarchical Community of Experts

SpeakerZhenwen Dai
AffiliationUniversity of Sheffield
DateFriday, 15 May 2015
Time13:00 - 14:00
LocationRoberts 508 (different than usual)
Event seriesDeepMind CSML Seminar Series

Deep latent variable models are promising for unsupervised and semi-supervised learning, however, the development of models with continuous latent variables are left behind. We scale up a deep continuous latent variable model called a hierarchical community of experts. It contains a hierarchy of linear-Gaussian units and a mechanism for dynamically selecting an subset of these units. We derive a new variational lower bound that only needs the estimation of the variational posterior at the top layer and use a probabilistic generative model for approximating such variational posterior by directly generating samples given the inputs, in which variance reduction techniques are not necessary. We verify our new variational bound and our inference generative model by applying to SBN, and compare the performance with the literature on the MNIST dataset. With training HCE on MNIST, we show that it is able to capture sophisticated variances of characters in generated covariance matrices.

iCalendar csml_id_228.ics