Research on robust decision making for intelligent connected vehicle at highway on-ramp

Reinforcement learning has demonstrated its potential in the decision-making field of autonomous driving. By engaging in trial-and-error learning and interacting with the environment, we can adapt our behavioral strategies based on reward and enhance driving decisions. Nevertheless, real-world vehicle decision-making involves intricate and diverse information, and the limited state information hinders agents from making optimal decisions, which may result in catastrophic consequences like collisions. Therefore, this paper proposes a robust deep reinforcement learning method based on the Soft Actor-Critic (SAC) algorithm for highway intelligent connected vehicle ramp merging decision-making. The model represents the vehicle features of the target lane and its adjacent lanes as an environmental state space. Additionally, a hybrid action space is designed, which combines discrete lateral actions and continuous longitudinal actions. Finally, an on-ramp simulation platform is built using actual roads and SUMO to verify the feasibility of the model. Multiple sets of comparative analysis experimental results demonstrate that the proposed method outperforms others in terms of the average reward return value, robustness value, collision rate, and success rate. Moreover, the model exhibits a stable lane change success rate under various road traffic density conditions, which indicates its robustness. The code can be obtained at https://github.com/ColinFanghz/sac-on-ramp.git .