AGP-MADDPG: A* Guidance Point based Multi-Agent Deep Deterministic Policy Gradient for 3D cooperative multi-UAV trajectory planning

Cooperative trajectory planning for multi unmanned aerial vehicle (multi-UAV) systems in three-dimensional (3D) environments is hindered by sparse rewards, long-horizon decision making, and unstable inter-agent coordination. To address these issues, an A* Guidance Point based Multi-Agent Deep Deterministic Policy Gradient (AGP-MADDPG) framework is developed, which integrates planning-based prior guidance with attention-based value estimation. In this framework, an improved A* planner with multi-objective constraints generates global reference paths by jointly considering path length, safety margin, and inter-UAV separation. Subsequently, dynamic guidance points are constructed along the reference path according to the UAVs’ positions, converting sparse feedback into dense guidance rewards to enhance exploration efficiency and training stability. To mitigate input dimensionality explosion and irrelevant interaction noise in the centralized critic, we incorporate a multi-head self-attention module. This module adaptively aggregates cross-agent information, improving both accuracy and scalability of value estimation. Simulations in basic and extended scenarios show that AGP-MADDPG consistently outperforms baselines in convergence speed, mission success rate, decision efficiency, and trajectory quality, demonstrating its effectiveness and robustness in cooperative multi-UAV trajectory planning.