Deep reinforcement learning for thermal comfort and energy-efficient vehicle thermal management
Yuan Pang, Kuining Li, Shuai Dai, Yi Xie, Dongpeng Zhao, Chi Huang
Abstract
Optimizing the control of vehicle thermal management systems is crucial for enhancing cabin comfort and improving energy efficiency. However, traditional control strategies such as rule-based and PID controllers have limited ability to adapt to dynamic and uncertain driving conditions. Relying on fixed logic and parameter settings, they struggle to handle the nonlinear and coupled characteristics of thermal management systems, resulting in suboptimal performance in balancing thermal comfort and energy efficiency. This paper addresses these limitations by presenting a model-free optimal control method based on the Double Deep Q-Network (DDQN), designed to manage the settings of the air conditioning system and cool/warm gear. The proposed approach aims to strike a balance between energy consumption and thermal comfort under varying environmental and operational conditions. Firstly, a one-dimensional simulation model of the thermal management system in a gasoline vehicle is established, encompassing vehicle dynamics, engine cooling, and the cabin air conditioning system. The control problem is then formulated as a Markov Decision Process (MDP), and four deep reinforcement learning (DRL) algorithms are compared. A novel DDQN-based control method is developed to optimize the control trajectories of the compressor, blower, and temperature settings. Compared to the baseline PID strategy, the proposed method reduces energy consumption by 11.5 % and decreases non-comfortable time by 30.9 %. Furthermore, the controller demonstrates robust performance across different ambient temperatures and test cycles, highlighting its adaptability and generalization capability.