Adaptive network traffic control with approximate dynamic programming based on a non-homogeneous Poisson demand model

In this study, we develop a stochastic dynamic traffic-flow model subject to practical restrictions under the non-homogeneous Poisson vehicle arrival process. Using the cell transmission strategy, we establish traffic dynamics between two intersections. We also discuss simulating the random demand of source links from an estimated intensity function. Additionally, we propose an algorithm to optimize time interval division for aggregated data, aiming to enhance estimation performance. We explore applying our traffic flow model to the adaptive traffic network management problem, which is formulated as a Markov decision process. Leveraging approximate dynamic programming with recursive least squares-temporal difference learning, we achieve adaptive optimal policies. To validate our approach, we conduct a series of numerical experiments with random demands. The results of non-homogeneous Poisson demand conducted using random numbers and a real-word dataset indicate high efficiency with the piecewise constant, I-SMOOTH, and MNO-PQRS estimators. Compared to the Webster and Max-pressure control systems, our proposed approximate dynamic programming-based model exhibits superior stability and applicability.