Continuous Reinforcement Learning-Based Energy Management Strategy for Hybrid Electric-Tracked Vehicles
Ruoyan Han, Renzong Lian, Hongwen He, Xuefeng Han
Abstract
The hybrid electric-tracked vehicles (HETVs) are usually used in both agricultural and industrial applications, while the optimal energy management is critical to fully exploit the potential of HETVs. In this article, the influence of HETVs’ steering resistance on the energy distribution is specially considered to model the dynamic demand accurately. Further, a multi-state energy management strategy (EMS) based on deep deterministic policy gradient (DDPG) is proposed for a series HETV in the continuous space. A multidimensional matrix framework is proposed to extract the parameters of the actor network from a trained DDPG-based EMS. Hardware-in-the-loop (HiL) experiment is conducted to validate the real-time tractability of the proposed strategy. Results suggest that the DDPG-based strategy improves the fuel economy remarkably by 13.1% and shows a more robust performance, compared with the double deep <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> -learning-based strategy. Though the proposed strategy is trained based on the fixed state of charge (SOC), it still exhibits a strong adaptability to the uncertainty of initial SOCs.