Routing Optimization With Deep Reinforcement Learning in Knowledge Defined Networking

Traditional routing algorithms cannot dynamically change network environments due to the limited information for routing decisions. Meanwhile, they are prone to performance bottlenecks in the face of increasingly complex business requirements. Some approaches, such as deep reinforcement learning (DRL) have been proposed to address the routing problems. However, they hardly utilize the information about the network environment fully. The Knowledge Defined Networking (KDN) architecture inspires us to develop new learning mechanisms adapted to the dynamic characteristics of the network topology. In this paper, we propose an effective scheme to solve the routing optimization problem by adding a graph neural network (GNN) structure to DRL, called Message Passing Deep Reinforcement Learning (MPDRL). MPDRL uses the characteristics of GNN to interact with the network topology environment and extracts exploitable knowledge through the message passing process of information between links in the topology. The goal is to achieve the load balance of network traffic and improve network performance. We have conducted experiments on three Internet Service Provider (ISP) network topologies. The evaluation results show that MPDRL obtains better network performance than the baseline algorithms.