Deep Reinforcement Learning Based Integrated Eco-driving Strategy for Connected and Automated Electric Vehicles in Complex Urban Scenarios

With vehicle-to-everything (V2X) information, connected and automated vehicle (CAV) eco-driving strategy allows the vehicle to plan its speed and choose the optimal lane based on actual conditions, resulting in improved driving performance. This study presents a novel eco-driving strategy framework based on deep reinforcement learning (DRL) techniques for CAVs driving in urban scenarios. This framework integrates longitudinal speed planning with lateral lane change decision-making and aims to co-optimize the energy efficiency, driving safety, and travel efficiency. By leveraging traffic information and multi-objective reward functions, the twin delayed deep deterministic (TD3) algorithm is employed to train the actor-critic (AC) network which generates both longitudinal and lateral control commands based on its estimation for lane preference score. The proposed strategy is tested in a complex urban scenario based on Simulation Urban Mobility (SUMO) which reflects real urban traffic conditions. Experimental results indicate that the longitudinal speed planning module of the proposed strategy can shorten the travel time by up to 7.94% or reduce the electricity consumption by 18.15%, depending on the degree of importance placed on economy by the TD3 agent. By integrating the lateral lane decision module, the proposed strategy can further shorten the travel time by 5.7% and save 1.75% energy consumption.