Federated Deep Reinforcement Learning-Based Multi-UAV Navigation for Heterogeneous NOMA Systems

The nonorthogonal multiple access (NOMA) technique for addressing fifth-generation (5G) new radio services is emerging as a promising technology. In contrast to the traditional orthogonal multiple access (OMA) techniques, NOMA can transmit a signal to various devices from the same resource block by exploiting the power domain and decoding the desired signal using the successive interference cancellation (SIC) technique. However, in large-scale 5G macro-cell scenarios, NOMA can need help to efficiently serve far-end devices due to long-distance transmission links, multipath fading, and non-line-of-sight (NLOS) problems. We explore flying small base stations (BSs), which can act as relays to mitigate link quality-related issues. Using the NOMA technique, multiple small-scale flying BSs can fly over the affected devices simultaneously and serve them without utilizing fewer resources compared to the conventional OMA method. We develop a novel intelligent unmanned aerial vehicle (UAV)-based navigation solution using federated deep reinforcement learning (DRL) whereby cost-efficient multiple UAV-BSs fly over certain areas autonomously and serve ground devices (GDs) using 5G NOMA systems without any interruption. We consider a greedy policy (GP) as a benchmark, where the UAV-BSs fly over high-priority devices to serve them. In addition, we consider the traveling salesman problem (TSP)-based navigation solution as a benchmark. We perform extensive simulation analysis for different system parameters, i.e., coverage time, coverage score (CS), and channel-to-noise ratio (CNR), and conclude that the proposed scheme outperforms other state-of-the-art methods.