Seminar: Hierarchical Bayesian Nonparametric Models for Sequences

SpeakerJan Gasthaus
DateFriday, 12 Oct 2012
Time12:30 - 14:00
LocationCruciform B404 - LT2
Event seriesDeepMind CSML Seminar Series

Hierarchical Bayesian nonparametric models based on the Dirichlet process (DP) or the Pitman-Yor process (PYP) have recently become popular because they provide a flexible framework for expressing prior beliefs over sets of related probability measures. One area where this approach has been particularly effective is sequence modeling in general and language modeling (i.e. modeling sequences of words in natural language text) in particular, where the dependencies between context-dependent probability distributions can naturally be modeled using a context tree hierarchy, and the power-law properties of the PYP prior match those found in natural language data. I will present the basic hierarchical PYP model, its extension to infinitely deep context trees (dubbed the "Sequence Memoizer"), and recent developments for modeling multi-domain data and non-stationary sequences.

Slides for the talk: PDF

iCalendar csml_id_72.ics