Computing Stabilizing Linear Controllers via Policy Iteration
Andrew Lamperski
Abstract
In recent years, a wide number of theoretical papers have focused on reinforcement learning approaches to the linear quadratic regulator (LQR) problem. However, nearly all of these papers assume that an initial stabilizing controller is given. This paper gives a model-free, off-policy reinforcement learning algorithm for computing a stabilizing controller for deterministic LQR problems with unknown dynamics and cost matrices. When the system is stabilizable, a controller which is guaranteed to stabilize the system is computed after finitely many steps. Furthermore, the solution converges to the optimal LQR gain.
Topics & Concepts
Linear-quadratic regulatorReinforcement learningControl theory (sociology)Controller (irrigation)Computer scienceOptimal controlQuadratic equationLinear systemMathematical optimizationMathematicsControl (management)Artificial intelligenceMathematical analysisGeometryAgronomyBiologyAdaptive Dynamic Programming ControlAdvanced Control Systems OptimizationReinforcement Learning in Robotics