A multi-objective hierarchical deep reinforcement learning algorithm for connected and automated HEVs energy management
Serdar Coskun, Ozan Yazar, Fengqi Zhang, Li Lin, Cong Huang, Hamid Reza Karimi
Abstract
Connected and autonomous vehicles have offered unprecedented opportunities to improve fuel economy and reduce emissions of hybrid electric vehicle (HEV) in vehicular platoons. In this context, a hierarchical control strategy is put forward for connected HEVs. Firstly, we consider a deep deterministic policy gradient (DDPG) algorithm to compute the optimized vehicle speed using a trained optimal policy via vehicle-to-vehicle communication in the upper level. A multi-objective reward function is introduced, integrating vehicle fuel consumption, battery state-of-the-charge, emissions, and vehicle car-following objectives. Secondly, an adaptive equivalent consumption minimization strategy is devised to implement vehicle-level torque allocation in the platoon. Two drive cycles, HWFET and human-in-the-loop simulator driving cycles are utilized for realistic testing of the considered platoon energy management. It is shown that DDPG runs the engine more efficiently than the widely-implemented Q-learning and deep Q-network, thus showing enhanced fuel savings. Further, the contribution of this paper is to speed up the higher-level vehicular control application of deep learning algorithms in the connected and automated HEV platoon energy management applications. • Proposing a hierarchical DDPG-based DRL algorithm to HEV platoon energy management. • Constructing a multi-objective reward function for HEV optimal speed planning policy. • Presenting a driving simulator-obtained results to show the real-time potentials. • Illustrating superiority of the DDPG-coordinated CACC algorithm over QL/DQN methods.