DRL-based Trajectory Planning for UAV-Assisted Communication Systems

Owing to the inherent superiority in high mobility and flexibility, unmanned aerial vehicles (UAVs)-enabled communication has emerged as a novel paradigm to enhance signal strength and coverage by serving as moving aerial relays. However, the existing schemes usually focus on one-way communication manner and are inflexible for changing environments and user number. To address these issues, we present a two-way UAV-enabled communication system when the practical factors, such as energy-constraint, limited resource and the potential interference source and obstacles, are considered jointly, and a joint optimization problem of trajectory and time allocation factor is formulated to maximize system throughput. To solve the formulated problem efficiently, the time-allocation factor is first derived in closed form, thereby reducing the original joint optimization to trajectory design. Then, a scalable deep reinforcement learning (DRL)-based framework with a fixed-dimensional state representation and a two-stage reward mechanism is developed, and the soft actor–critic (SAC) algorithm is adopted to optimize the UAV trajectory. Simulation results with varying settings demonstrate that our proposed scheme can improve system throughput significantly. It is also validated that the proposed scheme has better performance in convergence speed and robustness compared to the benchmark schemes.