Soft Actor-Critic Reinforcement Learning Approach to Multi-Drone 3D Terrain Scanning and Target Detection in Search and Rescue Operations

Unmanned Aerial Vehicle use has increased in many civil operations, such as search and rescue missions. As these vehicles become essential in more applications, the need to control and coordinate swarms of them efficiently increases. Search and rescue missions are usually carried out in difficult, dynamic, and complex scenarios where the chances of success rely on how well and fast the unmanned vehicles navigate and interact with these environments. Reinforcement Learning has been widely used in training UAVs for different tasks, as they require less prior knowledge than other Machine Learning methods of their environment. This paper’s main contribution is the use of the recent Soft Actor-Critic method to train swarms of drones in terrain scanning and target detection through increasingly complex terrains, from sparse scenarios to densely populated obstacle fields. Unlike other methods, SAC encourages exploration and adaptability through an entropy term, making it suitable for target location in unknown areas. The model is trained in a simulated environment with a reward system that prioritizes drone coordination to cover new areas and find the most targets, while avoiding obstacles and area limits. Sensing capabilities of drones vary according to their flight altitude, so they learn to balance between larger area coverage and more precise detection. The model is then compared to other navigation algorithms, and it shows that it can be more efficient than them on average, finding 58% of targets in the first hour, and 75% of them in the second hour. Out of the other path strategies explored, the best alternative (Random Walk) only found targets with 46% and 73% success in the first and second hours, respectively. The preliminary results show that the model can be further improved, since the model is not reliable enough to enhance operations significantly at its current state. Future research can focus on implementing Soft-Actor Critic with other algorithms for hybrid approaches that enhance performance; training and testing the model in new environments, or including fault detection systems in drones.