Coordinated Operation of Multiple Microgrids With Heat–Electricity Energy Based on Graph Surrogate Model-Enabled Robust Multiagent Deep Reinforcement Learning

The control of heat-electricity-integrated multiple microgrid (MMG) systems is greatly challenged by anomalous measurements and inaccurate physical electricity and heat network models. Through the systematic integration of graph surrogate models, trajectory history information, and confederate image (CI) technology based distributed multiagent deep reinforcement learning (MADRL), we propose a robust coordinated control approach for the optimization of MMG systems. Each MG in the MMG system is first represented as a graph with tree topology that is processed by a graph neural network (GNN)-based module to produce robust representations of the measurements. Subsequently, the GNN-based module produces information that is fed into a fully connected layers module to model realistic power and thermal flow using historical data in a supervised manner, thereby forming the graph surrogate models. Before the MADRL training, the GNN-based module from trained surrogate models is embedded in the policy network of MADRL. With the support of CI, the state information and information from the GNN-based module are proceeded by the extracting trajectory history feature module. This process endows the MADRL-based controller with the ability to identify and correct anomalous measurements. The information from the GNN-based module further enhances the robustness against anomalous measurements. The trained surrogate models provide the reward signal to MADRL during MADRL training. It enables the proposed approach to be independent on accurate MMG parameter estimates. The effectiveness of the proposed approach is validated by the simulation results.