Multi-Scale Reinforcement Learning of Dynamic Energy Controller for Connected Electrified Vehicles
Hao Zhang, Nuo Lei, Shengbo Eben Li, Junzhi Zhang, Zhi Wang
Abstract
The synergy of reinforcement learning (RL)-based energy management and vehicle-to-everything communication has been proved effective in boosting the fuel economy of connected plug-in hybrid electric vehicles (PHEVs). However, the intricate coupling of mechanical, electrical, thermal states and driving cycle results in a high-dimensional complex energy control problem for PHEVs, which is challenging to optimally solve within the same time scale. To this end, this study designs a multi-horizon reinforcement learning (MHRL)-based energy management of PHEVs, aware of the traffic preview from intelligent transportation systems to optimize the energy flow and thermal states as well as the transient dynamics of the powertrain. The proposed strategy features a novel state space representation, and solves the coordinated training among multiple sub-networks belonging to different control tasks in various time scales. Simulation and hardware-in-the-loop experiments are carried out based on a standard driving cycle and a real-world driving cycle with real-time traffic data demonstrate that the MHRL strategy improves fuel economy by 3.0%~7.9% compared to conventional RL-based energy management under various coolant temperature conditions and dynamic driving scenarios.