Seminar: Lifting, VarPro, ICP, and all that.

SpeakerAndrew Fitzgibbon
AffiliationMicrosoft Research Cambridge
DateFriday, 10 Jun 2016
Time13:00 - 14:00
LocationRoberts G06 Sir Ambrose Fleming LT
Event seriesDeepMind CSML Seminar Series
Description

Abstract:
In vision and machine learning, from 3D reconstruction to recommender systems, it is common to see optimization problems of the form

$\min_x \sum_i \min_u f_i(x,u)$

There are a few main strategies for minimizing these problems: block coordinate descent (a.k.a. alternation, “EM-style”, or ICP), joint optimization (a.k.a. lifting or bundle-style), variable projection (VarPro), and the various SGD techniques. For years I have been using lifting to great effect, and I will show examples where it dramatically improves convergence rates and wall-clock speed. Recently, new light has been cast on these alternatives, and I will show examples where VarPro wins hands down. Ultimately, I’ll try to give intuitions that allow you to know into which case your problem falls and when it matters; that is, when it’s important to use the more advanced strategies rather than ICP or SGD.

Joint work with John Hong, Cambridge University, and many others.

iCalendar csml_id_291.ics