Seminar: Evaluating generative models

SpeakerLucas Theis
DateFriday, 20 Apr 2018
Time13:00 - 14:00
LocationRoberts Building G08 Sir David Davies LT
Event seriesDeepMind CSML Seminar Series

Probabilistic generative models can be used for compression, denoising, inpainting, texture synthesis, semi-supervised learning, unsupervised feature learning, and other tasks. Given this wide range of applications, it is not surprising that a lot of heterogeneity exists in the way these models are formulated, trained, and evaluated. As a consequence, direct comparison between models is often difficult. In this talk, we are going to take a look at some of the metrics which have been used to evaluate generative models. In particular, we will see that three popular criteria – average log-likelihood, Parzen window estimates, and visual fidelity of samples – are largely independent of each other when the data is high-dimensional. Good performance with respect to one criterion therefore need not imply good performance with respect to the other criteria. We conclude that generative models need to be evaluated directly with respect to the application(s) they were intended for.

iCalendar csml_id_345.ics