Combining Markov Decision Processes with Linear Optimal Controllers
Subject
Management Science and Operations
Publishing details
Authors / Editors
Abramova E;Kuhn D;Faisal A
Biographies
Publication Year
2011
Abstract
Linear Quadratic Gaussian (LQG) control has a known analytical solution [1] but non-linear problems do not [2]. The state of the art method used to find approximate solutions to non-linear control problems (iterative LQG) [3] carries a large computational cost associated with iterative calculations [4]. We propose a novel approach for solving nonlinear Optimal Control (OC) problems which combines Reinforcement Learning (RL) with OC. The new algorithm, RLOC, uses a small set of localized optimal linear controllers and applies a Monte Carlo algorithm that learns the mapping from the state space to controllers. We illustrate our approach by solving a non-linear OC problem of the 2-joint arm operating in a plane with two point masses. We show that controlling the arm with the RLOC is less costly than using the Linear Quadratic Regulator (LQR). This finding shows that non-linear optimal control problems can be solved using a novel approach of adaptive RL.
Publication Notes
Reinforcement Learning; non-linear Optimal Control; locally linear approximations; Linear Quadratic Regulator; robotic arm
Available on ECCH
No
