A Hierarchical Conflict Resolution Framework With Graph Transformer-Based Reinforcement Learning for Heterogeneous UAV Networks

Dense unmanned aerial vehicle (UAV) networks comprising different types of UAVs have been widely applied across various domains, leading to potential conflicts within heterogeneous UAV networks. To ensure UAV safety, effective conflict resolution is crucial. However, existing learning-based methods are primarily designed for homogeneous UAVs and struggle to learn effective maneuvers for UAVs with varying attributes (like velocity and safety radius) or diverse motion pattern, such as hovering. To address these challenge, we propose a hierarchical framework with graph transformer-based reinforcement learning (HGT-RL) for conflict resolution in heterogeneous UAV networks. Specifically, a novel heterogeneous graph transformer network is employed to enhance the extraction of heterogeneous node information through orderly embeddings while incorporating a graph transformer network. To reduce the training complexity for heterogeneous UAVs, we generate the heuristic maneuvers at each time step, with the reinforcement learning model trained solely to refine these initial actions or perform emergency hovering. Experimental results demonstrate that HGT-RL outperforms previous methods in heterogeneous UAV networks, and we further validate its scalability across multiple scenarios.