Deep-Reinforcement-Learning-Based Energy Management Strategy for Supercapacitor Energy Storage Systems in Urban Rail Transit

The modeling complexity of the traction power system and variation of traffic conditions bring challenges for the optimization of energy management strategy for supercapacitor energy storage systems in urban rail transit. Therefore, in this paper a deep-reinforcement-learning-based energy management strategy is proposed: the energy management system is modeled as an intelligent agent, the reward function is formulated comprehensively considering the energy-saving and voltage-stabilizing effects of supercapacitor, a traction power system simulator is developed to emulate the environment, and the agent's behavior is improved in each headway through the deep Q-learning algorithm, and converges to the nearly-optimal policy. The proposed strategy is verified through simulation based on the Beijing Subway Batong Line. The study results show that it dynamically adjusts the voltage thresholds so as to better allocate the supercapacitor capacity along the time horizon. The energy-saving and voltage-stabilizing effects are significantly improved compared with the fixed-threshold strategy and genetic optimization, and demonstrating to be in close proximity to the optimal benchmark deduced from dynamic programming.