Enabling efficient routing for traffic engineering in SDN with Deep Reinforcement Learning

Traffic Engineering (TE) is applied to optimize network transmission efficiency by managing the routing of complicated traffic. Emerging Deep Reinforcement Learning (DRL) and Software-Defined Networking (SDN) provide flexible ability for traffic management and congestion control. However, existing methods either cannot reroute the network-wide traffic in an accurate way or encounter too much calculation latency due to the inference of mathematical optimization techniques. In this paper, we propose a flexible and link-reconfigurable TE solution called EfficientTE that effectively adjusts the traffic routing based on real-time traffic demands. By analyzing the characteristics of network topology and traffic, EfficientTE selects a few links that are critical for congestion in the network. Then, we propose the idea of virtual capacity that helps the DRL algorithm adjust to different link bandwidths. Based on the traffic demand and topology information collected by the SDN controller, the DRL algorithm is used to dynamically adjust the virtual capacity of the critical links to reshape the network. To ensure network performance with low disturbance, we selectively reroute the Top-K critical flows using the weighted K-shortest path algorithm, while forwarding the major flows with default rerouting. Experiments show that EfficientTE optimizes maximum link utilization and outperforms existing TE solutions by improving the load-balancing performance ratio by at least 6.13%, 18.78%, 16.20%, and 22.81% respectively in four network topologies.