Hierarchical Reinforcement Learning-Based Distributed Fault-Tolerant Control for 2-D Plane Vehicular Platoon Systems

This paper investigates the distributed fault-tolerant control (DFTC) problem for two-dimensional (2-D) plane vehicular platoon systems (VPS) with asynchronous steering and spacing requirements. Each vehicle in the VPS is subject to time-varying unknown direction actuator faults (UDAF) and unknown uncertainties, including nonlinear unmodeled dynamics and external disturbances. These challenges are addressed by the proposal of a novel hierarchical reinforcement learning-based DFTC framework with two layers. In the high-level path planning layer, vehicles within the VPS obtain the navigation positions from the paths planned by conditional deep Q-networks in conjunction with predecessor information and designed spacing projection constraints. The navigation positions are updated in real-time to satisfy asynchronous steering and spacing requirements, ensuring communication connectivity and collision avoidance. In the low-level DFTC layer, an online distributed fault-tolerant tracking controller is constructed by integrating a Nussbaum function with actor-critic neural networks (NNs). This controller effectively mitigates the impact of time-varying UDAF, while the actor NN compensates for bias faults and unknown uncertainties under the guidance of the critic NN. Simulation results in the Carla simulator validate the effectiveness and superiority of the proposed DFTC algorithm in lane merging and steering scenarios of 2-D plane VPS.