Scalable reinforcement learning for large-scale coordination of electric vehicles using graph neural networks
Stavros Orfanoudakis, Valentin Robu, E. Mauricio Salazar, Peter Pálenský, Pedro P. Vergara
Abstract
As the adoption of electric vehicles (EVs) accelerates, addressing the challenges of large-scale, city-wide optimization becomes critical in ensuring efficient use of charging infrastructure and maintaining electrical grid stability. This study introduces EV-GNN, a novel graph-based solution that addresses scalability challenges and captures uncertainties in EV behavior from a Charging Point Operator's (CPO) perspective. We prove that EV-GNN enhances classic Reinforcement Learning (RL) algorithms' scalability and sample efficiency by combining an end-to-end Graph Neural Network (GNN) architecture with RL and employing a branch pruning technique. We further demonstrate that the proposed architecture's flexibility allows it to be combined with most state-of-the-art deep RL algorithms to solve a wide range of problems, including those with continuous, multi-discrete, and discrete action spaces. Extensive experimental evaluations show that EV-GNN significantly outperforms state-of-the-art RL algorithms in scalability and generalization across diverse EV charging scenarios, delivering notable improvements in both small- and large-scale problems.