Longitudinal Control of Automated Vehicles: A Novel Approach by Integrating Deep Reinforcement Learning With Intelligent Driver Model

Deep reinforcement learning (DRL) provides a promising approach for the implementation of autonomous driving. By utilizing a trained DRL model as the longitudinal controller, the automated vehicle (AV) can generate optimal action outputs based on the state within a shorter time compared to traditional model predictive control (MPC) methods. However, the non-interpretability of neural networks poses a potential risk for real-world vehicle operation. This paper focuses on applying the Twin Delayed Deep Deterministic Policy Gradient (TD3), a state-of-the-art (SOTA) DRL algorithm, to train the longitudinal control model for AVs. We confirm the risks associated with the TD3-based longitudinal control model by assessing its violation of the rational driving constraint (RDC), which represents the basic conditions for normal driving behaviors. To mitigate these risks, we propose a novel model that integrates the TD3-based model with the intelligent driver model (IDM) using a new indicator called velocity response time (VRT). This indicator identifies risky outputs of the TD3-based model and calculates the combined weights of both the IDM and TD3-based models. This combination allows us to reduce risks associated with the non-interpretability of the neural network while also capturing the effect of engine time lag. Numerical simulations are conducted to evaluate the performance of the proposed combined model. The results demonstrate that the proposed combined model outperforms the TD3-based model, IDM, and another SOTA approach in terms of disturbance mitigation, safety improvement, and suppression of traffic oscillation. Additionally, the combined model exhibits greater computational efficiency than MPC, making it well-suited for real-time control of AVs.