Deep reinforcement learning based energy management strategy for multi-mode hybrid electric vehicle with dual planetary gear set

Energy Management Strategy (EMS) plays a pivotal role in enhancing the fuel economy of hybrid electric vehicles. Deep Reinforcement Learning (DRL), as a cutting-edge optimization technique, holds remarkable potential for improving EMS. However, traditional DRL control algorithms encounter limitations when dealing with the complex and unique discrete-continuous hybrid action space inherent to dual planetary gear hybrid systems. To address this challenge, this paper innovatively proposes a DRL algorithm based on the Parameterized Deep Q-Network (P-DQN). This algorithm can simultaneously regulate the clutch’s operational state and the engine’s output speed. Through parameterized action design, we effectively convert continuous actions into discrete parameterized representations, enabling precise simultaneous control over discrete and continuous actions. Based on P-DQN, we further introduce the Multi-Pass Q-network structure to optimize the neural network. This structure severs the correlation between irrelevant parameters and the Q-value, thereby enhancing the accuracy and efficiency of the control strategy. We denominate this refined algorithm as Multi-Pass Deep Q-Network(MP-DQN). To validate the effectiveness of the MP-DQN algorithm, we conducted comprehensive comparative experiments against P-DQN and other typical DRL-based methods such as DDPG. The results demonstrate significant fuel economy improvements with the MP-DQN-based EMS, proving the effectiveness and superiority of the proposed approach.