Joint Trajectory and Beamforming Optimization for UAV-RIS-Empowered Multiuser Communication Networks: A Double Deep Q-Network Approach

Reconfigurable intelligent surface (RIS) assisted unmanned aerial vehicle (UAV) communication technology offers adaptable reflected beams and dynamic parameter adjustments, proving vital for managing the highly dynamic communication environments expected in wireless communications. However, a key challenge lies in addressing the dynamic and nonlinear problem of RIS phase shift design and UAV trajectory prediction. To ensure long-term fairness and enhance system performance, we propose an approach based on the double deep Q-network (DDQN), aiming to maximize the weighted sum rate across multiple users. Specifically, we employ an initial phase shift and predefined codebook to optimize the RIS phase shift at the current UAV position using DDQN. Further refinement is achieved through an extremum seeking algorithm. Then, under constraints encompassing RIS phase shift, allowable regions for UAV node, base station (BS) transmit power, and minimum user rate, we adopt DDQN to obtain predefined reward parameters jointly designed by fairness and throughput through interaction with dynamic and variable channel environment, and continuously update UAV position parameters to maximize rewards. The simulation results verify the efficacy of our joint optimization strategy, demonstrating optimal UAV location selection for UAV-RIS-empowered multiuser networks in a 2D plane.