Sum Rate Maximization for RIS-assisted UAV-IoT Networks using Sample Efficient SAC Technique

Deep Reinforcement Learning (DRL) based algorithms have been widely adopted to solve the non-convex optimization problems in Reconfigurable Intelligent Surface (RIS)assisted Unmanned Aerial Vehicle (UAV) systems for establishing uninterrupted wireless connections with the ground Internet of Things (IoT) devices. However, model-free DRL techniques such as Deep Deterministic Policy Gradient (DDPG), Deep Q-learning (DQN) and Double Deep Q-learning (DDQN) suffer from low convergence and poor sample efficiency. Use of off-policy DRL techniques can be an appropriate solution to enhance the sample efficiency and training speed in vulnerable and fast changing environments involving multiple IoTs. In this paper, we use a novel off-policy actor-critic DRL technique called Soft ActorCritic (SAC) to solve the sum rate maximization problem in RIS-assisted UAV-IoT networks in dense urban environment. We perform simulations to compare the results of the proposed sample efficient SAC algorithm with the existing DDPG technique with and without RIS optimization. Our simulations show that SAC with optimized RIS outperforms the model-free DDPG technique in terms of maximizing the sum rate.