Multiobjective Trajectory Planning for UAV-Assisted IoT Networks Based on DRL Approach

Uncrewed aerial vehicles (UAVs), due to their inherent flexibility and autonomous operation, are widely used in Internet of Things (IoT) networks for collecting data to facilitate real-time evaluation and monitoring applications. This article investigates a UAV-assisted IoT network, in which the UAV sequentially accesses IoT devices (IoTDs). During hovering, the UAV works on a full-duplex mode while collecting data from target devices, taking into account actual propulsion power consumption. In order to quantify the freshness of devices data, we introduce the concept of Age of Information (AoI). A multiobjective optimization method is proposed to jointly optimize three objectives: 1) maximization of the data rate; 2) minization of AoI; and 3) minization of the UAV energy consumption over a particular mission period. These three objectives partially conflict with each other and provide weight parameters to describe their importance. Since the data uploaded by IoTDs is dynamically changing, the trajectory planning of the UAV is required. Considering that the UAV has no prior knowledge of the network environment, the optimization problem is reformulated as a Markov decision process. Aiming at the learning problem of the UAV control strategies over multiple objectives, a deep reinforcement learning algorithm for multiobjective collaborative optimization is proposed. While training, the agent collects data in time according to the devices priority, and generates optimal strategies under the conditions of giving weights. Experimental results demonstrate that the proposed multiobjective twin delayed deep deterministic policy gradient algorithm jointly optimizes three objectives and can adjust the optimal policy based on the weight parameters of each objective.