Securing UAV Communication Based on Multi-Agent Deep Reinforcement Learning in the Presence of Smart UAV Eavesdropper

In this paper, we investigate an unmanned aerial vehicle (UAV)-enabled secure communication system, where ground nodes send confidential information to a legitimate UAV by time division multiple access in the presence of a smart UAV eavesdropper. It is a practical scenario that the UAV eavesdropper will make full use of its mobility for more effective eavesdropping and its trajectory can not be obtained in advance. Firstly, the problem of maximizing the sum secrecy rate by jointly optimizing the legitimate UAV trajectory, transmit power control and node scheduling is formulated from perspective of legitimate UAV. Next, due to the presence of smart eavesdropper and time-varying environment caused by uncontrollable mobility of UAV eavesdropper, we reformulated the original problem as a two-player zero-sum stochastic game (TZSG) problem. In order to solve the TZSG problem, considering competitive scenario and the mixed action space of the TZSG, we propose an algorithm based on multi-agent deep reinforcement learning to obtain a policy of legitimate communication link nodes and simulation results verify the proposed algorithm has superior performance than benchmark algorithm.