Imitation and Transfer Learning for LQG Control
Taosha Guo, Abed AlRahman Al Makdah, Vishaal Krishnan, Fabio Pasqualetti
Abstract
In this letter we study an imitation and transfer learning setting for Linear Quadratic Gaussian (LQG) control, where (i) the system dynamics, noise statistics and cost function are unknown and expert data is provided (that is, sequences of optimal inputs and outputs) to learn the LQG controller, and (ii) multiple control tasks are performed for the same system but with different LQG costs. We show that the LQG controller can be learned from a set of expert trajectories of length <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$n$ </tex-math></inline-formula> (l+2)−1, with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$n$ </tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$l$ </tex-math></inline-formula> the dimension of the system state and output, respectively. Further, the controller can be decomposed as the product of an estimation matrix, which depends only on the system dynamics, and a control matrix, which depends on the LQG cost. This data-based separation principle allows us to transfer the estimation matrix across different LQG tasks, and to reduce the length of the expert trajectories needed to learn the LQG controller to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$2n+m-1$ </tex-math></inline-formula> with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$m$ </tex-math></inline-formula> the dimension of the inputs (for single-input systems with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$l=2$ </tex-math></inline-formula> , this yields approximately a 50% reduction of the required expert data).