A Hierarchical Spatio-Temporal Cooperative Reinforcement Learning Approach for Traffic Signal Control

The traffic signal control is expected to flexibly control the phase sequence and duration based on dynamic traffic flow in each direction, enabling efficient traffic efficiency and reducing congestion. To this end, in this paper, we propose a hierarchical spatio-temporal collaboration reinforcement learning (HSTCRL) algorithm, which achieves fine-grained control to each phase with flexible duration. For the basic layer, long short-term memory (LSTM) and graph attention mechanism (GAT) are introduced to model the temporal and spatial dependence of traffic flow respectively, enabling intersections to choose phase cooperatively. For the high layer, double DQN is used to adjust phase duration based on information from the intersection of the entire region, avoiding insufficient or too long green light time. The experiment results have demonstrated the effectiveness of HSTCRL, which shows better traffic performance than the state-of-the-art methods.