Intelligent User-Centric Network Selection: A Model-Driven Reinforcement Learning Framework

Ultra-dense heterogeneous networks, as a novel network architecture in the fifth-generation mobile communication system (5G), promise ubiquitous connectivity and smooth experience, which take advantage of multiple radio access technologies (RATs), such as WiFi, UMTS, LTE, and WiMAX. However, the dense environment of multi-RATs challenges the network selection because of the more frequent and complex decision process along with increased complexity. Introducing artificial intelligence to ultra-dense heterogeneous networks can improve the way we address network selection today, and can execute efficient and intelligent network selection. Whereas, there still exist difficulties to be noted. On one hand, the contradiction between real-time communications and time-consuming learning is exacerbated, which can result in slow convergence. On the other hand, the black-box learning mode suffers from oscillation due to the diversity of multi-RATs, which can result in arbitrary convergence. In this paper, we propose a model-driven framework with a joint off-line and on-line way, which is able to achieve fast and optimal network selection through an alliance of machine learning and game theory. Further, we implement a distributed algorithm at the user side based on the proposed framework, which can reduce the number of frequent switching, increase the possibility of gainful switching, and provide the individual service. The simulation results confirm the performance of the algorithm in accelerating convergence rate, boosting user experience, and improving resource utilization.