Multi-Agent Reinforcement Learning for Intelligent V2G Integration in Future Transportation Systems

Electric vehicles (EVs) are the backbone of the future intelligent transportation system (ITS). They are environmentally friendly and can also be integrated as distributed energy resources (DERs) into the smart grid using vehicle-to-grid (V2G) scheme. Specifically, utility companies can push back EV batteries into the electric grid to reduce the peak load. However, integrating EVs into the power grid efficiently requires accurate artificial intelligence (AI) mechanisms to forecast, coordinate, and dispatch the EVs into the grid. This paper proposes a Multi-agent Reinforcement Learning (MARL) mechanism that schedules the day-ahead discharging process of EV batteries to optimize the peak shaving performance of the electric grid. The proposed MARL overcomes the inaccuracy of energy prediction by allowing the agents, i.e. EVs, to make autonomous decisions. These agents are trained in a centralized fashion but make decisions locally to maintain autonomy and privacy. In particular, the model does not require that the EVs communicate with a centralized entity during the execution stage, which assures the model's integrity and protects the EVs' private information. To evaluate the model, a comprehensive series of experiments were carried out to prove the effectiveness of the MARL coordination and scheduling mechanism and to show that the model can indeed flatten the peak load.