Topology Planning Using Q-Learning for Microwave-Based Wireless Backhaul Networks

Today, a microwave-based solution is the one commonly used for wireless backhaul networks as it has high capacity and can be easily deployed. For such a microwave-based wireless backhaul network, a well-designed topology is important for efficient capacity utilization and high-quality mobile services. In an earlier work, we presented an approach for topology planning for microwave-based wireless backhaul networks, where an integer linear programming (ILP) model was formulated and a heuristic algorithm was also developed, subject to various constraints. However, the previous heuristic algorithm may merely lead to local optima, and moreover, only a single set of weight factors was considered for the optimization. This cannot guarantee joint optimization over all the relevant performance aspects. We overcome these drawbacks here by employing the Q-learning technique for this topology planning problem. This consists of (a) re-optimizing an initial topology based on a specific set of weight factors and (b) finding an optimal set of these weight factors. To accelerate the Q-learning process, we also develop a parallel Q-learning system to find the optimal set of weight factors. Simulation results indicate that the Q-learning based approach can jointly optimize multiple system objectives and outperforms our earlier heuristic algorithm. The parallel learning system can also significantly expedite the learning process.