Seminar: Lifting, VarPro, ICP, and all that.

SpeakerAndrew Fitzgibbon
AffiliationMicrosoft Research Cambridge
DateFriday, 10 Jun 2016
Time13:00 - 14:00
LocationRoberts G06 Sir Ambrose Fleming LT
Event seriesDeepMind CSML Seminar Series

In vision and machine learning, from 3D reconstruction to recommender systems, it is common to see optimization problems of the form

$\min_x \sum_i \min_u f_i(x,u)$

There are a few main strategies for minimizing these problems: block coordinate descent (a.k.a. alternation, “EM-style”, or ICP), joint optimization (a.k.a. lifting or bundle-style), variable projection (VarPro), and the various SGD techniques. For years I have been using lifting to great effect, and I will show examples where it dramatically improves convergence rates and wall-clock speed. Recently, new light has been cast on these alternatives, and I will show examples where VarPro wins hands down. Ultimately, I’ll try to give intuitions that allow you to know into which case your problem falls and when it matters; that is, when it’s important to use the more advanced strategies rather than ICP or SGD.

Joint work with John Hong, Cambridge University, and many others.

iCalendar csml_id_291.ics