Trajectory and Beamforming Optimization in UAV-enabled ISAC System

A multiple unmanned aerial vehicles (UAVs) enabled integrated sensing and communication (ISAC) system is investigated. In contrast to existing UAV-enabled ISAC systems assuming static users or 2D UAV trajectory, we consider a practical roaming user scenario and a 3D deployment for UAVs. Then, a joint trajectory and beamforming optimization problem is formulated for maximizing the long-term sum data rate, subject to the transmitting power constraint and ensuring beam pattern gain constraint for sensing target. We proposed a two-step approach for against the dynamic scenario: 1) a K-means based hierarchical user association algorithm is proposed to renew the user association periodically. 2) a hybrid reward multi-agent proximal policy optimization (HR-MAPPO) algorithm is proposed, which decomposes the complex combined reward into a team reward and an individual reward. Numerical results demonstrate that the proposed HR-MAPPO algorithm can outperform conventional single-agent and multi-agent RL algorithms by maintaining high scores on both sum data rate and beam pattern gain.