Cooperative Control Model Using Reinforcement Learning for Connected and Automated Vehicles and Traffic Signal Light at Signalized Intersections

Effectively leveraging data and domain knowledge remains a significant challenge in controlling the Internet of Unmanned Agent (IUA). This article proposes a novel multiagent deep reinforcement learning-based cooperative control model called MARL-CTV to efficiently control two key IUA agents: 1) connected and automated vehicle (CAV) and 2) controllable traffic signal light (TSL). The CAV agents are controlled by the deep deterministic policy gradient (DDPG) algorithm, and the TSL agent is controlled by a dueling double deep Q-network (D3QN). To reduce the control burden and ensure the cumulative reward converges, the actor and critic networks are pretrained by the expert dataset, and the expert dataset initializes the experience replay buffer of DDPG. This dataset is generated by multiple velocity profiles derived from a genetic algorithm (GA) based on various random initial states of CAVs. Numerical experiments conducted using a joint simulation platform composed of SUMO and CARLA and real-world data from CitySim demonstrate the effectiveness of MARL-CTV. Specifically, when the market penetration rate (MPR) of CAV is 35%, MARL-CTV enables most CAVs to pass through the signalized intersection without stop-and-go behavior, reducing average travel time by 24.2%, fuel consumption by 22.7%, and the traffic conflicts by 68.3%.