Scalable reinforcement learning for large-scale coordination of electric vehicles using graph neural networks

Abstract As the adoption of electric vehicles (EVs) accelerates, addressing the challenges of large-scale, city-wide optimization becomes critical in ensuring efficient use of charging infrastructure and maintaining electrical grid stability. This study introduces EV-GNN, a novel graph-based solution that addresses scalability challenges and captures uncertainties in EV behavior from a Charging Point Operator’s (CPO) perspective. We prove that EV-GNN enhances classic Reinforcement Learning (RL) algorithms’ scalability and sample efficiency by combining an end-to-end Graph Neural Network (GNN) architecture with RL and employing a branch pruning technique. We further demonstrate that the proposed architecture’s flexibility allows it to be combined with most state-of-the-art deep RL algorithms to solve a wide range of problems, including those with continuous, multi-discrete, and discrete action spaces. Extensive experimental evaluations show that EV-GNN significantly outperforms state-of-the-art RL algorithms in scalability and generalization across diverse EV charging scenarios, delivering notable improvements in both small- and large-scale problems.