Hierarchical, Heterogeneous Control of Non-Linear Dynamical Systems using Reinforcement Learning
Journal
Journal of Machine Learning Research
Subject
Management Science and Operations
Publishing details
Authors / Editors
Abramova E;Dickens L;Kuhn D;Faisal A
Biographies
Publication Year
2012
Abstract
Non-adaptive methods are currently state of the art in approximating solutions to nonlinear optimal control problems. These carry a large computational cost associated with iterative calculations and have to be solve individually for different start and end points. In addition they may not scale well for real-world problems and require considerable tuning to converge. As an alternative, we present a novel hierarchical approach to non-Linear Control using Reinforcement Learning to choose between Heterogeneous Controllers, including localised optimal linear controllers and proportional-integral-derivative (PID) controllers, illustrating this with solutions to benchmark problems. We show that our approach (RLHC) competes in terms of computational cost and solution quality with state-of-the-art control algorithm iLQR, and offers a robust, flexible framework to address large scale non-linear control problems
Keywords
Non-linear dynamics; reinforcement learning; hierarchical control; optimal control; LQR control; PID control; robotic arm control; cart on pole swing up & control
Available on ECCH
No