Urban Traffic Control in Software Defined Internet of Things via a Multi-Agent Deep Reinforcement Learning Approach

As the growth of vehicles and the acceleration of urbanization, the urban traffic congestion problem becomes a burning issue in our society. Constructing a software defined Internet of things(SD-IoT) with a proper traffic control scheme is a promising solution for this issue. However, existing traffic control schemes do not make the best of the advances of the multi-agent deep reinforcement learning area. Furthermore, existing traffic congestion solutions based on deep reinforcement learning(DRL) only focus on controlling the signal of traffic lights, while ignore controlling vehicles to cooperate traffic lights. So the effect of urban traffic control is not comprehensive enough. In this article, we propose Modified Proximal Policy Optimization (Modified PPO) algorithm. This algorithm is ideally suited as the traffic control scheme of SD-IoT. We adaptively adjust the clip hyperparameter to limit the bound of the distance between the next policy and the current policy. What's more, based on the collected data of SD-IoT, the proposed algorithm controls traffic lights and vehicles in a global view to advance the performance of urban traffic control. Experimental results under different vehicle numbers show that the proposed method is more competitive and stable than the original algorithm. Our proposed method improves the performance of SD-IoT to relieve traffic congestion.