Deep Reinforcement Learning for Data Freshness-oriented Scheduling in Industrial IoT

Making a timely and precise scheduling in Industrial Internet of Things (IIoT) is fundamental and critical. Recently, Age of Incorrect Information (AoII) is proposed and utilized to measure the timeliness and accuracy of monitoring. In this work, we investigate a multi-sensor update system and leverage AoII to quantify the information freshness. Our goal is to obtain an optimal scheduling policy to minimize the system-wide cost. We first model the source statuses monitored by sensors as Markov chains and the scheduling problem as a Markov decision process (MDP). Due to the heterogeneity of source statuses in IIoT, it is prohibitive to solve the formulated MDP problem by conventional methods. To this end, we make use of a deep reinforcement learning (DRL) algorithm to solve this scheduling problem. Extensive numerical results verify the effectiveness of the adopted DRL algorithm. In addition, comparing to the conventional Age of Information (AoI) oriented method, we find that the AoII oriented method is much more effective, from the perspective of system-wide cost.