Spatiotemporally Decoupled Hierarchical Reinforcement Learning for Cooperative Multi-UAV Task Allocation and Path Planning

To address the curse of dimensionality and heterogeneous resource conflicts in cooperative multi-UAV task allocation and path planning, this paper proposes a spatiotemporally decoupled Hierarchical Reinforcement Learning (HRL) framework. The framework separates long-horizon discrete scheduling from short-horizon continuous navigation planning. In the upper layer, an improved Q-Mixing Network (QMIX) is developed with a multi-constraint Stacking Mask to prune infeasible actions and a SumTree-based prioritized experience replay mechanism to mitigate sparse rewards, enabling efficient heterogeneous task allocation. In the lower layer, a hybrid navigation strategy integrating Soft Actor-Critic (SAC) and Artificial Potential Field (APF) is designed. SAC learns continuous navigation actions to avoid unstructured static obstacles, while APF provides local dynamic compensation for formation coordination and safe collision avoidance. High-fidelity AirSim simulations demonstrate that the proposed framework achieves effective task allocation and safe path execution in distributed heterogeneous and high-priority saturated task scenarios.