Cooperative Network Model for Joint Mobile Sink Scheduling and Dynamic Buffer Management Using Q-Learning

Development of energy-efficient wireless sensor networks is crucial in the deployment of IoT and IIoT for modern day applications like smart home, smart vehicles, and smart industries. Several methods like network clustering, mobile sink deployment and dynamic sensing rate have been used in improving the energy-efficiency of wireless sensor networks in IoT framework. However, these methods have been developed independently which can lead to certain network issues like reduced lifetime, network breakdown among others. In this work, an energy-efficient method that optimizes mobile sink scheduling while concurrently providing dynamic buffer management is proposed. A cooperative network model that incorporates node clustering and mobile sink deployment in variable node sensing rate scenario is first developed. However, in such cooperative network models, mobile sink scheduling and buffer overflow management which causes information loss become challenging. This is primarily due to limited buffer size, variable sensing rate of the nodes, and the unavailability of mobile sink at all times near a cluster. Therefore, a reinforcement Q-learning framework is developed for scheduling the mobile sink while minimizing the information loss caused by buffer overflow in each cluster of a clustered WSN. More specifically, the network behaviour is learnt in the context of buffer overflow using Q-learning approach. The proposed method computes the adaptive halt-times for the mobile sink based on information loss and buffer overflow in each cluster. Performance of the proposed joint mobile sink scheduling and dynamic buffer management method is evaluated on a medium scale WSN. A clustered wireless sensor network with a total of 600 sensor nodes is considered for performance evaluation. The proposed method is shown to learn the variable node sensing rate in a reasonable amount of time using convergence analysis. Numeric evaluations indicate that the proposed method minimizes the information loss in a medium scale wireless sensor network while improving the network lifetime simultaneously. The proposed cooperative network model also outperforms in terms of energy-efficiency when compared to conventional WSN. The results are motivating enough for the use of cooperative network model in practical WSNs for IoT applications.