A deep reinforcement learning-based active suspension control algorithm considering deterministic experience tracing for autonomous vehicle

As the challenges in autonomous driving become more complex and changing, traditional methods are struggling to cope. As a result, artificial intelligence (AI) techniques have gained widespread attention due to their potential in addressing these challenges. To investigate the application and performance of deep reinforcement learning (DRL) techniques in vertical control of autonomous vehicles, we propose an active suspension control algorithm that incorporates deterministic experience tracing (DET). The agent explores and learns deterministic policies by interacting with the environment and continuously exploring and exploiting the generated data. During this process, DET stores state and action data in a separate experience memory buffer over time. Additionally, DET processes this information into auxiliary rewards that decay based on temporal logic. This drives the agent to self-iterate and rapidly improve. DET allows AI techniques to incorporate temporal robustness into data-driven learning, resulting in improved generalization performance and optimized ride comfort in engineering applications. Simulation results demonstrated that DET improved control performance by 74.92%, 64.20%, and 54.64% compared to the deep deterministic policy gradient (DDPG), twin delayed deep deterministic policy gradient (TD3), and model predictive control (MPC) baselines, respectively. Furthermore, it achieved nearly a 90% improvement in ride comfort on random roads in classes A, B, and C across different speeds. Even on class D roads, the optimization remained around 85%, demonstrating its excellent generalization performance.