Cooperative Optimization on Computation, Communication and Trajectory for UAV Crowdsensing: A Multi-Agent Deep Reinforcement Learning Approach

The rapid growth of the Internet of Things (IoT) and consumer electronics (CEs) have underscored the importance of intelligent systems, which can efficiently manage and optimize resource in dynamic environments. Multiple unmanned aerial vehicles (UAVs) serving as intelligent agents to form the crowdsensing system for data collection is thus very promising, especially in areas with weak infrastructure and insufficient services. In this study, we investigate the joint optimization problem for trajectory design, resource allocation and computation offloading to maximize service data volume with users’ fairness in multi-UAV crowdsensing system. In particular, the joint optimization problem is modeled as a Decentralized Partial Observation Markov Decision Process (Dec-POMDP) to deal with its non-convex characteristics and the environmental dynamicity. To solve this problem, we then propose an Actor-Critic based online multi-agent deep reinforcement learning (MADRL) approach that integrates the heuristic strategy to enhance decision-making. Extensive simulation results demonstrate the convergence and efficiency of the proposed method, providing 11.86% 21.61% performance improvement compared to state-of-the-art methods.