Skip to main content

Please enter a keyword and click the arrow to search the site

Combining Markov Decision Processes with Linear Optimal Controllers


Management Science and Operations

Authors / Editors

Abramova E;Kuhn D;Faisal A

Publication Year



Linear Quadratic Gaussian (LQG) control has a known analytical solution [1] but non-linear problems do not [2]. The state of the art method used to find approximate solutions to non-linear control problems (iterative LQG) [3] carries a large computational cost associated with iterative calculations [4]. We propose a novel approach for solving nonlinear Optimal Control (OC) problems which combines Reinforcement Learning (RL) with OC. The new algorithm, RLOC, uses a small set of localized optimal linear controllers and applies a Monte Carlo algorithm that learns the mapping from the state space to controllers. We illustrate our approach by solving a non-linear OC problem of the 2-joint arm operating in a plane with two point masses. We show that controlling the arm with the RLOC is less costly than using the Linear Quadratic Regulator (LQR). This finding shows that non-linear optimal control problems can be solved using a novel approach of adaptive RL.

Publication Notes

Reinforcement Learning; non-linear Optimal Control; locally linear approximations; Linear Quadratic Regulator; robotic arm

Available on ECCH


Select up to 4 programmes to compare

Select one more to compare
subscribe_image_desktop 5949B9BFE33243D782D1C7A17E3345D0

Sign up to receive our latest news and business thinking direct to your inbox


Sign up to receive our latest course information and business thinking

Leave your details above if you would like to receive emails containing the latest thought leadership, invitations to events and news about courses that could enhance your career. If you would prefer not to receive our emails, you can still access the case study by clicking the button below. You can opt-out of receiving our emails at any time by visiting: or by unsubscribing through the link provided in our emails. View our Privacy Policy for more information on your rights.