Multiagent Deep Reinforcement Learning for Transactive Energy Management of MMGs Incorporating Battery Swapping Stations

Optimal energy management between microgrids (MGs) and battery swapping stations (BSSs) offers significant economic benefits. However, existing works face challenges in formulating optimal interaction strategies between MGs and BSSs, due to the temporal-spatial uncertainty of distributed renewables and emerging loads, as well as incomplete information. This article addresses the energy transaction problem between multi-MGs and multi-BSSs using a hybrid multiagent deep reinforcement learning approach to minimize operation costs. Specifically, a hierarchical transactive energy management community is introduced to facilitate energy exchange between MMGs and BSSs, considering different stakeholder interests. The problem is modeled as a partially observable Markov game. The proposed hybrid algorithm, combining multiagent proximal policy optimization (MAPPO) and double deep Q-network (DDQN), handles the continuous scheduling of MGs and the discrete operations of BSSs. Numerical results show that, averaged over the baselines, the proposed MAPPO-DDQN reduces 13.71% of MGs' operation costs and increases 14.62% of BSSs' profit.