Eco-Driving Strategy Design of Connected Vehicle Among Multiple Signalized Intersections Using Constraint-Enforced Reinforcement Learning

Optimizing speed profiles at urban signalized intersections, commonly referred to as an eco-driving strategy, is acknowledged as a promising approach to improving vehicle energy efficiency. However, the unpredictable nature of traffic signals and traffic flow can reduce the effectiveness of such eco-driving strategies. This paper proposes an eco-driving strategy based on constraint-enforced reinforcement learning (CE-RL) for connected and automated vehicles (CAVs) between multiple signalized intersections, taking into account the influence of preceding vehicles. First, an energy-efficient control problem is formulated to minimize energy consumption while considering driving safety and travel time. The driving speed is constrained by the optimal speed range for green lights. Second, a hierarchical learning-based control framework is proposed to solve the optimal control problem. The upper stage uses quadratic programming to derive feasible actions that satisfy the driving safety constraints, while the lower stage uses the RL algorithm to optimize the energy-efficient driving profile in a stochastic driving environment. Finally, simulation results show that the proposed CE-RL eco-driving strategy outperforms conventional eco-driving approaches in stochastic driving environments and provides adaptability to different driving scenarios. In addition, a field test is conducted to show that the proposed strategy is capable of reducing energy consumption in real driving situations.