Learning to Routing in UAV Swarm Network: A Multi-Agent Reinforcement Learning Approach

The past few years have witnessed an exponential growth of compelling UAV swarm applications ranging from agricultural production, and intelligent transport, to disaster rescue. The high-speed mobility of the UAV swarm belongs to a new clan of networks, termed flying ad-hoc networks (FANETs). How to design an effective routing mechanism in such a dynamic network is challenging. Traditional flooding searching algorithms (e.g., OLSR, AODV) lead to huge communication overheads, while greedy searching algorithms (e.g., Geolocation-Based Routing protocol) pose low routing efficiency. In this paper, we propose an adaptive communication-based UAV swarm routing algorithm. In our algorithm, we design the Multilayer Perceptron algorithm to learn when routing flooding is required among different UAVs, and design the Gate Recurrent Unit algorithm to greatly compress the volume of communication data. Besides, we further adopt the multi-agent actor-critic algorithm to learn how to integrate shared information for cooperative routing decision-making. Extensive simulation results validate that our algorithms achieve efficient and effective routing under a partially observable distributed environment for large-scale UAV swarm cooperation.