Hierarchical graph multi-agent reinforcement learning for traffic signal control

Multi-agent reinforcement learning (MARL) is a promising algorithm for traffic signal control (TSC), and graph neural networks make a further improvement on its learning capacity. However, these state-of-the-art algorithms adopt fine-granularity information (i.e., current step-states) to get traffic-network embeddings, but ignore great-granularity information (e.g., previous multiple step-states); In addition, these algorithms are optimized by loss functions in the viewpoint of MARL, but ignore the correlation between input information and output embeddings. This paper proposes a Hierarchical Graph Multi-agent Mutual Information (HG-M2I) algorithm for TSC. Specifically, HG-M2I algorithm is developed based on: 1) Multi-granularity fusion employs a proposed Hierarchical Graph representation learning (HG) algorithm, which fuses both current step-states of multiple agents and previous multiple step-states of each agent, thus facilitates to produce optimal final embeddings; 2) Joint optimization employs a proposed Multi-agent Mutual Information (M2I) framework, which measures the correlation between input step-states and output embeddings by maximizing mutual information, the corresponding mutual-information and MARL loss jointly optimize the whole algorithm, thus facilitates to derive optimal traffic-signal policies. HG-M2I is compared against these state-of-the-art algorithms on synthetic-road and real-road datasets. Experimental results not only show the superior performance of HG-M2I over other compared algorithms, but also illustrate its advantageous transferability.