Seminar: Combining state abstraction and temporal abstraction in MDP solving

SpeakerKamil Ciosek
AffiliationUCL, Computer Science
DateFriday, 16 Jan 2015
Time13:00 - 14:00
LocationRoberts G08 (Sir David Davies lecture theatre)
Event seriesMicrosoft Research CSML Seminar Series

The talk presents a way of solving Markov Decision Processes that
combines state abstraction and temporal abstraction. Specifically, we
combine state aggregation with the options framework and demonstrate
that they work well together and indeed it is only after one combines
the two that the full benefit of each is realized. We introduce a
hierarchical value iteration algorithm where we first coarsely solve
subgoals and then use these approximate solutions to exactly solve the
MDP. This algorithm solves several problems faster than vanilla value

About the speaker: Kamil Ciosek ( is a PhD student at CSML specialising in approximate approaches to solving MDPs.

iCalendar csml_id_209.ics