Destabilizing Attack and Robust Defense for Inverter-Based Microgrids by Adversarial Deep Reinforcement Learning

The controllers of inverter-based resources (IBRs) can be adjustable by grid operators to facilitate regulation services. Considering the increasing integration of IBRs at power distribution level systems like microgrids, cyber security is becoming a major concern. This paper investigates the data-driven destabilizing attack and robust defense strategy based on adversarial deep reinforcement learning for inverter-based microgrids. Firstly, the full-order high-fidelity model and reduced-order small-signal model of typical inverter-based microgrids are recapitulated. Then the destabilizing attack on the droop control gains is analyzed, which reveals its impact on system small-signal stability. Finally, the attack and defense problems are formulated as Markov decision process (MDP) and adversarial MDP (AMDP). The problems are solved by twin delayed deep deterministic policy gradient (TD3) algorithm to find the least effort attack path of the system and obtain the corresponding robust defense strategy. The simulation studies are conducted in an inverter-based microgrid system with 4 IBRs and IEEE 123-bus system with 10 IBRs to evaluate the proposed method.