Traffic Engineering in a Shared Inter-DC WAN via Deep Reinforcement Learning

To reduce the investment on network infrastructure, many online service providers have begun to adopt the shared inter-DataCenter Wide Area Network (inter-DC WAN) that connects different datacenters and Internet Service Providers (ISPs). The shared inter-DC WAN accommodates two kinds of traffic, i.e. delay-sensitive ISP-facing traffic and high-throughput inter-DC traffic. Traffic Engineering (TE) in the shared inter-DC WAN should determine the routing paths for all traffic to achieve link load balancing, while select lower-latency egress routers for ISP-facing traffic to guarantee the Quality of Service (QoS). Therefore, this paper mainly focuses on jointly optimizing routing paths selection and egress router selection to strike a balance between QoS and link load balancing. Specifically, we first formulate the TE problem in the shared inter-DC WAN as a mixed integer nonlinear programming problem. Then, a TED method is proposed to jointly optimize the egress router selection and routing path selection by learning an intelligent agent with Deep Reinforcement Learning (DRL). The learnt agent can self-adaptively and rapidly select the optimal egress routers by considering the utilization-latency balance when traffic demand changes. Finally, we conduct extensive evaluations on Alibaba WAN with real traffic traces to demonstrate the effectiveness and superiority of the proposed method.