A TD3 Algorithm Based Reinforcement Learning Controller for DC-DC Switching Converters
Jian Ye, Huanyu Guo, Sen Mei, Yingjie Hu, Xinan Zhang
Abstract
Various linear and nonlinear controllers have been developed to improve the dynamic performance of DC-DC converters. Most controllers can only be designed on the basis of understanding the mathematical model of DC-DC converter, but the inherent nonlinear and time-varying characteristics of DC-DC switching converter make it difficult to complete the precise modeling, so the model-based control design is complex and the control performance is limited. In order to overcome the problem, this paper proposes a reinforcement learning (RL) controller based on the twin-delayed deep deterministic policy gradient (TD3) algorithm. This controller does not need the model of the switching converter. The converter will be regarded as a black box model, the policy approximation function (policy neural network) can be trained and learned by constructing a Markov decision process interacting with the black box model in the control system, and the optimal control action can be output. The RL controller is developed based on actor critic architecture, and a TD3 algorithm with higher learning efficiency is proposed to improve the control performance of the RL controller. The proposed RL controller based on TD3 algorithm is compared with the traditional PI controller. The simulation results show that the RL controller has better dynamic performance when the converter starts and the load step changes.