Constrained large-scale real-time EV scheduling based on recurrent deep reinforcement learning

The rapid growth of electric vehicles (EVs) is an unstoppable worldwide development trend. An optimal charging strategy for large-scale EVs is able to deal with the randomness of EVs charging and satisfy charging demands of users while ensuring safe and economic operation of the power system. The current centralized and model-based methods failed to overcome the randomness charging problem of the large-scale EVs. Thus, the paper proposes a decentralized approach based on model-free deep reinforcement learning (DRL) to determine the optimal strategy for reducing EVs charging cost considering power limit of the charging station (CS), users' charging demands and fair charging fees. First, a decentralized framework and a dynamic energy boundary (DEB) model of single EV which discretizes the charging demand are proposed. Second, the problem as a Markov Decision Process (MDP) with unknown transition probability is formulated. Moreover, the recurrent deep deterministic policy gradient (RDDPG) based approach is proposed to determine the charging strategy for all charging piles in the CS. Finally, digital simulation studies are conducted to demonstrate the effectiveness of the proposed approach in charging cost reduction and fair charging fees. In addition, the RDDPG-based approach has great scalability which can apply a small-scale model to solve a large-scale problem without being retrained.