Deep Q-Learning-Based Mobile Charger Path Planning in Wireless Powered Communication Networks

Wireless Powered Communication Network (WPCN) is a new paradigm to allow low-power wireless devices to exchange data packets and receive stable energy transfer from a power source and thus support autonomous and sustainable network operations without battery replacements. In recent years, we have witnessed the growing deployment of WPCNs in both industrial and consumer IoT systems to support time-triggered and event-triggered monitoring applications. In this paper, we present a novel reinforcement learning (RL)-based on-demand path planning framework to plan the trajectory of a Mobile Charger (MC) and schedule the charging sequence of wireless devices to sustain the network operations. A modified Deep Q-learning approach is designed to charge the wireless devices by balancing between their residual energy level and the distance from the MC to the device. This approach minimizes the total distance that the MC travels while ensuring that individual residual energy of a given set of devices is above a designated threshold. Extensive experimental results from both the Gazebo-based high-fidelity simulation and Turtlebot-based physical testbed demonstrate that our approach outperforms the classic scheduling methods (e.g., Nearest Job Next and Earliest Deadline First), state-of-the-art scheduling methods(Extended Particle Swarm Optimization, Enhanced Teaching–Learning-Based Optimization Algorithm and Spatiotemporal Optimization for Charging Scheduling), learning-based methods (e.g., Proximal Policy Optimization and Advantage Actor-Critic) with similar sample sizes for training.