High robustness energy management strategy of hybrid electric vehicle based on improved soft actor-critic deep reinforcement learning

As a hybrid electric vehicle (HEV) key control technology, intelligent energy management strategies (EMSs) directly affect fuel consumption. Investigating the robustness of EMSs to maximize the advantages of energy savings and emission reduction in different driving environments is necessary. This article proposes a soft actor-critic (SAC) deep reinforcement learning (DRL) EMS for hybrid electric tracked vehicles (HETVs). Munchausen reinforcement learning (MRL) is adopted in the SAC algorithm, and the Munchausen SAC (MSAC) algorithm is constructed to achieve lower fuel consumption than the traditional SAC method. The prioritized experience replay (PER) is proposed to achieve more reasonable experience sampling and improve the optimization effect. To enhance the “cold start” performance, a dynamic programming (DP)-assisted training method is proposed that substantially improves the training efficiency. The proposed method optimization result is compared with the traditional SAC and deep deterministic policy gradient (DDPG) with PER through the simulation. The result shows that the proposed strategy improves both fuel consumption and possesses excellent robustness under different driving cycles. • A robust energy management strategy is established based on the SAC algorithm. • Munchausen reinforcement learning method is adopted to the SAC algorithm. • Prioritized experience replay is applied to improve training efficiency. • DP-assisted training method is proposed to enhance the “cold start” performance. • The proposed framework realizes better performance in fuel-saving and robustness.