Reinforcement Learning-Based Formation Control for Uncrewed Surface Vehicles Under Aperiodic DoS Attacks: A Stackelberg–Nash Game Approach

This article investigates the distributed formation control of uncrewed surface vehicles (USVs) under aperiodic denial-of-service (DoS) attacks within a Stackelberg-Nash game (SNG) framework. An actor-critic (AC) reinforcement learning (RL) algorithm is developed to approximate these policies online, ensuring convergence to the Stackelberg-Nash equilibrium (SNE). To enhance resilience against communication interruptions, a consensus-based estimator is designed to reconstruct missing neighbor data using local information. Rigorous Lyapunov-based analysis guarantees the input-to-state stability (ISS) of the estimator and the semi-globally uniformly ultimately bounded (SGUUB) stability of the closed-loop system. Simulation results verify the framework's effectiveness in achieving accurate trajectory tracking and robustness against frequent DoS attacks.