Channel Selection and Power Control for D2D Communication via Online Reinforcement Learning

In this paper, we address the problem of Device-to-Device communication (D2D) in the next generations of cellular networks, where the number of D2D pairs can grow large and, hence, improving spectral efficiency becomes a crucial design factor. More specifically, decentralized channel selection and power control by D2D pairs for interference mitigation without inflicting a heavy controlling overhead on the network become significant challenges in allocating resources. To this end, we introduce an online distributed reinforcement learning algorithm at D2D pairs to maximize network throughput, while guaranteeing both D2D users’ and cellular users’ (CUs) Quality of Service (QoS) under the dynamic wireless channel environment. To track and evaluate the performance of the proposed online algorithm, we define three metrics, i.e., D2D collision probability, D2D access rate and time-average network throughput. The simulation results confirm the convergence property of the proposed algorithm and shows improved performance in terms of three defined metrics as compared to the celebrated Q-learning-based method.