Intelligent Trajectory Design for Mobile Energy Harvesting and Data Transmission

Energy harvesting technology enables wireless sensor networks (WSNs) to be self-sustainable, for maintaining long-term key performance indicators, such as the data throughput and sensing coverage. Due to the highly dynamic and complex environment, energy sources (ES) cannot provide stable energy supply, which needs the efficient learning algorithm to enable system adaptations. This article reports on the development of reinforcement learning (RL) methodology to long-term data collection in self-sustainable WSNs. Specifically, we consider the WSN as a 2-D rectangular region, where a mobile sensor (MS) can harvest energy from ambient environments while transmitting the collected data to a fixed sink. Due to the changing environment and the mobility of the MS, the harvested energy by the MS at each slot presents spatiotemporal dynamics within the network, which severely affects the performance of data throughput from the MS to the sink. The MS's trajectory is investigated to maximize the long-term average MS-to-sink data throughput. Due to the unknown energy arrival information as well as the locations of ESs, we formulate the problem as a Markov decision process, which is then solved with model-free RL. In particular, the deep deterministic policy gradient (DDPG) is applied to tackle the continuous and deterministic movement space. Results show that the MS can learn and optimize the moving trajectory by intelligently tracking the aggregated received energy over slots. Finally, the MS can identify and move to the optimal location where the maximized long-term average MS-to-sink data throughput is achieved. Extensive numerical evaluations are conducted to investigate the impact of various system parameters on the network performance.