AAV Visual Navigation in the Large-Scale Outdoor Environment: A Semantic-Map-Based Cognitive Escape Reinforcement Learning Method

The rapid development of the autonomous aerial vehicle (AAV) technology has significantly facilitated its application in the burgeoning Internet of Things (IoT) ecosystem. AAV visual navigation has emerged as a particularly vibrant and crucial research area, holding the potential to greatly enhance automated IoT services. deep reinforcement learning (DRL), recognized as an effective method for visual navigation, encounters two significant challenges in the complex outdoor environments, i.e., the partial observability and the local optima trapping. In this article, to address these two challenges, we propose a novel semantic map-based cognitive escape reinforcement learning (SM-CERL) navigation method, which consists of two innovatively designed modules, namely the semantic mapping module (SMM) and the cognitive escape module (CEM). By deeply exploring the similarity structure of raw images and mapping them to the advanced semantic representations, the SMM constructs a semantic map with rich implications, which provides the AAV with the memory to enhance its understanding of the environment. Meanwhile, the CEM can proactively identify local optima and leverage the rule knowledge to establish efficient escape strategies. The holistic fusion of semantic mapping and cognitive escape mechanisms efficiently enhances the environmental comprehension and prevents AAVs from being trapped in local optima. Extensive experimental results demonstrate that our SM-CERL outperforms the existing classical and state-of-the-art methods in terms of the navigation accuracy and efficiency.