A Note on State Parameterizations in Output Feedback Reinforcement Learning Control of Linear Systems
Syed Ali Asad Rizvi, Zongli Lin
Abstract
This note presents an analysis of the state parameterizations used in output feedback reinforcement learning (RL) control. Output feedback algorithms based on state parameterization involve additional conditions on the state parameterization beyond the standard conditions on the system matrices for their convergence to the optimal solution. It is shown that the state parameterization matrix needs to be of full row rank to guarantee the convergence of the output feedback RL algorithms. We present conditions in terms of the system matrices and the user-defined observer dynamics that ensure full row rank of the state parameterization matrix.
Topics & Concepts
Convergence (economics)Control theory (sociology)State (computer science)Reinforcement learningRank (graph theory)Observer (physics)Computer scienceState observerMatrix (chemical analysis)Linear systemOutput feedbackControl (management)Mathematical optimizationMathematicsAlgorithmArtificial intelligenceNonlinear systemQuantum mechanicsCombinatoricsMaterials scienceEconomicsEconomic growthMathematical analysisPhysicsComposite materialAdaptive Dynamic Programming ControlReinforcement Learning in RoboticsAdaptive Control of Nonlinear Systems