Overlapping Coalition Formation-Enabled Noncooperative Game-Combined Multi-Agent DRL for UAV-Assisted Resource Allocation

Coalition Formation (CF) game emerges as a pioneering framework for the resource allocation in Unmanned Aerial Vehicles (UAVs) equipped with various types and complementary resources. However, both the overlapping-enabled collaborative CF and inter-coalition competitive behaviors significantly affect the system performance in complex multi-UAV scenarios. In this paper, we propose a Multiple Overlapping Coalitions (MOC) noncooperative game. Specifically, we first establish an optimization model encompassing coupled resource constraints. Subsequently, a task-priority-based incentive mechanism is designed to better motivate participation. To achieve the Nash equilibrium, a two-step solution technique incorporating relaxation and fine-tuning of resource granularity is designed. We propose a MOC noncooperative game-combined Multi-agent Proximal Policy Optimization (MAOPPPO). The simulation results substantiate that our approach outperforms the other five state-of-the-art learning countermeasures in terms of average reward with a gain of up to$4.59\%$after 800 of training episodes. In terms of throughput, the proposed MOC noncooperative game increases by$66.67\%$,$93.68\%$, and$11.76\%$compared with that of CF noncooperative game, non-CF noncooperative game, and consensus-based algorithm, respectively. For total resource contribution, the improvements are$62.99\%$,$94.59\%$, and$23.16\%$, respectively. The energy efficiency enhances by$6.82\%$,$23.68\%$, and$4.78\%$compared to the other three baselines, respectively.