Centre for Computational Statistics and Machine Learning

Stochastic optimal control theory provides a principled answer to the problem of computing an optimal sequence of actions to reach a goal in the presence of uncertainty. The solution is based on dynamic programming and known as the Bellman equation. However, the actual computation is typically very costly and scales exponentially in the dimension of the problem.

Recently, it was shown that a quite large class of non-linear control problems could be solved using an alternative approach using a diffusion process. The optimal control can be represented as a path integral, an expectation over future trajectories. This solution can be computed much more efficiently, using MCMC. In this talk, I will introduce the theory and give some examples. In addition, I show how the path integral control formalism can be used for learning the optimal control of a deterministic plant. I will illustrate the approach with some examples.