Safe Path Planning of UAV Based on Reinforcement Learning in Probabilistic Environments

Safety is a critical concern for flights in probabilistic environments of autonomous Unmanned Aerial Vehicles (UAVs). The traditional model-based path planning methods for probabilistic environments are often computationally demanding. When using model-free Reinforcement Learning (RL), an action with a higher risk could be chosen if the performance to the long-term goal is high enough. This paper will incorporate a probabilistic safety constraint to the model with Q-Learning that will dynamically adjust the collision penalty to achieve the required collision probability threshold. To train and test the proposed safe-RL model in an uncertain environment, a probabilistic collision space map is introduced where the collision areas are determined from a data-driven distribution. With the probabilistic environments, the goal of RL is to maximize the expected reward while maintaining an expected low collision chance. This model is then further tested on a realistic city map with a flow simulation around a group of urban buildings, where areas of high velocity are considered as probabilistic collision spaces and will be safety-critical for urban flights.