A Novel Bi-Level Framework for Electric Vehicle Charging Scheduling Using Reinforcement Learning and Transfer Learning

The growing adoption of electric vehicles (EVs) presents significant challenges for efficiently managing charging infrastructure, a critical component in the transition toward sustainable transportation. This study addresses the Electric Vehicle Charging Scheduling Problem (EVCSP), a computationally complex optimization problem aimed at maximizing the number of satisfied charging demands while adhering to grid and infrastructure constraints. To address this challenge, a novel hybrid bi-level optimization framework is proposed. At the upper level, advanced reinforcement learning (RL) methods, specifically Q-learning, Double Q-learning, SARSA, and Expected SARSA, are employed to generate adaptive EV-to-charger assignment policies by iteratively exploring and learning from decision environments. These assignments, which may not fully satisfy operational constraints, are subsequently refined at the lower level through a mathematical programming model that optimizes energy allocation under realistic grid capacity and charger availability constraints. Additionally, transfer learning is incorporated to enhance computational efficiency by transferring knowledge gained during the RL-based assignment process to the optimization phase. Extensive experimental evaluations demonstrate the framework’s scalability and effectiveness, with SARSA combined with transfer learning emerging as the most reliable method, particularly for larger and more complex problem instances.