DRL-ECMS: An Adaptive Hierarchical Equivalent Consumption Minimization Strategy Based on Deep Reinforcement Learning

With the rise of machine learning, reinforcement learning (RL) is gradually applied to the energy management strategy (EMS) of plug-in hybrid electric vehicle (PHEV). Some old algorithms have also achieved better results by combining with reinforcement learning. In order to learn from the advantages of previous algorithms and explore the application potential of reinforcement learning algorithm, this paper proposes an adaptive hierarchical management strategy combining equivalent consumption minimization strategy (ECMS) knowledge with proximal policy optimization (PPO). This system is an advanced data-driven RL algorithm at present. For a more comprehensive comparison, this paper compares the proposed EMS with dynamic programming (DP), ECMS with constant equivalence factor and q-learning. The results show that the fuel consumption of the proposed control strategy is very close to that of the DP-based control strategy and the performance is better than the other two strategies. It shows that deep reinforcement learning can help ECMS solve the problem of dynamic factor planning and DRL-ECMS has the potential of deployment in real-time system.