TransComm-MARL: A Transformer-Based MultiAgent Reinforcement Learning Approach for NonGround UAV-Satellite Networks

This paper proposes a hybrid non-terrestrial network architecture integrating drones and satellites, combining radio frequency (RF) and free-space optical (FSO) links. All drones are centrally controlled via control links to optimize energy consumption and system throughput. To address this communication scenario, we introduce a Transformer-based multi-agent reinforcement learning (MARL) algorithm, TransComm-MARL. The encoder extracts internal representations from joint observations, while the decoder employs an autoregressive approach for sequential decision-making. A masking mechanism ensures that each agent considers only the decisions of its predecessor, thereby reducing the complexity of joint decision-making. Experiments show that TransComm-MARL outperforms baseline models, achieving a $93.57 \%$ reduction in energy consumption and a $31.77 \%$ increase in throughput after convergence.