Reinforcement Learning-Based Dynamic Beam Hopping and Power Allocation for RSMA-Assisted Constrained-Input Satellite System

This paper investigates a dynamic beam hopping and power allocation problem in rate-splitting multiple access (RSMA) assisted multi-beam satellite system, in which beam services are allocated for terrestrial cells with non-uniform traffic demands and spectrum resource is full-frequency reused among beams for serving multiple ground terminals through RSMA technique. The objective is to jointly optimize beam hopping and power allocation strategy for maximizing the long-term sum rate while simultaneously satisfying delay fairness demand. However, the interference caused by full frequency reuse among beams limits the efficacy of RSMA, and the frequent beam patterns changes make instantaneous channel state information (CSI) detection a challenge. To address these challenges, this paper designs a novel deep reinforcement learning (DRL) based resource allocation framework for discrete-continuous action spaces. To alleviate explosion of action space, each beam is treated as a DRL agent, and a multi-agent deep reinforcement learning method is proposed to learn the beam scheduling policy. In addition, an action mapping mechanism is employed to further manage the DRL power allocation for satisfying the power input constraint arising from limited satellite power budget. Simulation results demonstrate that the proposed method can achieve the highest sum rate of the system while ensuring fairness among non-uniform traffic load cells compared with several baseline methods.