Health-Consciousness Integrated Thermal and Energy Management of Connected Hybrid Electric Vehicles Using Cooperative Multi-Agent Deep Reinforcement Learning
Arash Khalatbarisoltani, Jie Han, Wenxue Liu, Congzhi Liu, Xiaosong Hu
Abstract
Integration of energy management strategies (EMSs) with intelligent transportation systems exhibits a promising way to use the energy-saving potential of hybrid electric vehicles (HEVs). Battery, engine, cabin, and their heating and cooling systems are examples of dynamic thermal loads that typically exist during HEV operation. These loads are variable and highly influenced by ambient temperature and driving conditions. The primary objective of this paper is to develop a decentralized health-consciousness learning-based integrated thermal and energy management (ITEM) system that, besides minimizing fuel consumption, considers driver comfort and battery lifetime in the context of speed and temperature preview. In this regard, a multi-agent deep reinforcement learning (MADRL) framework with long short-term memory (LSTM) is proposed to consider the distinct dynamic characteristics of the ITEM system as well as capture the influence of past observations. A high-fidelity power-split HEV model established by Autonomie is utilized to implement and analyze the performance of the proposed approach. Based on the results, the multi-agent method outperforms both the rule-based and single-agent strategies. The MADRL agent can reduce battery degradation by 48% compared to rule-based while keeping the cabin temperature in the comfort zone. Furthermore, the proposed multi-agent strategy is implemented in a hardware-in-the-loop (HIL) system to evaluate its efficacy in real-world scenarios. Our experimental results demonstrate that the battery temperature and cabin temperature curves closely match the simulation results, with a maximum absolute error of 0.45 and 0.85, respectively.