Model-free Maneuvering Control of Fixed-Wing UAVs Based on Deep Reinforcement Learning

Maneuvering control of fixed-wing unmanned aerial vehicles (UAVs) is widely regarded to be a highly challenging control problem. As the maneuvering flight requires time-varying high dynamical control, it is particularly difficult to design controllers that achieve such a high performance. This paper aims to handle the S-snakes maneuver for fixed-wing UAVs with model-free deep reinforcement learning (DRL). The policy network has the multiple continuous states input and multiple continuous action output, and we extend the Gate Recurrent Unit (GRU) block to reduce the impact of low Markov characteristics in control task. A new method using Soft Actor Critic (SAC) with Proportion Differentiation (PD) teacher guidance is proposed to accelerate the training process for this complex network. In the simulation of handling F16 aircraft, this intelligence control method achieve a much better tracking performance than the PD control method.