Expert-demonstration-augmented reinforcement learning for lane-change-aware eco-driving traversing consecutive traffic lights

Eco-driving methods incorporating lateral motion exhibit enhanced energy-saving prospects in multi-lane traffic contexts, yet the randomly distributed obstructing vehicles and sparse traffic lights pose challenges in assessing the long-term value of instantaneous actions, impeding further improvement in energy efficiency. In response to this issue, a deep reinforcement learning (DRL)-based eco-driving method is proposed and augmented with the expert demonstration mechanism. Specifically, a Markov decision process matching with the target eco-driving scenario is systematically constructed, with which, the formulated DRL algorithm, parametrized soft actor-critic (PSAC), is trained to realize the integrated optimization of speed planning and lane-changing maneuver. To promote the training performance of PSAC under sparse rewards concerning traffic lights, an expert eco-driving model and an adaptive sampling approach are incorporated to constitute the expert demonstration mechanism. Simulation results highlight the superior performance of the proposed DRL-based eco-driving method and its training mechanism. Compared with the performance of the PSAC with a pure exploration-based training mechanism, the expert demonstration mechanism promotes the training efficiency and cumulated rewards of PSAC by about 60 % and 21.89 % respectively in the training phase, while in the test phase, a further reduction of 4.23 % benchmarked on a rule-based method is achieved in fuel consumption.