Multiagent Deep Reinforcement Learning for Electric Vehicle Fast Charging Station Pricing Game in Electricity-Transportation Nexus

Transportation electrification, involving large-scale integration of electric vehicles (EV) and fast charging stations (FCS), constitutes one of the key enablers toward decarbonization. Coordination of EV charging routes and demand through suitably designed price signals constitutes an imperative step in secure and economic operation of the coupled transportation network (TN) and power distribution network (PDN). In this work, we model the noncooperative pricing game of self-interested FCSs, taking into account the complex interactions between the EV users and the coupled operation of TN and PND. The uncertainties stemming from the EV users' cost elasticity and their travel energy requirements are encapsulated in the modeling of the TN, while the power flows in the PDN are coordinated considering the penetration of renewable energy sources. A modified multiagent proximal policy optimization method is developed to solve the pricing game. It employs an attention mechanism to selectively incorporate agents' representative information for estimating the Q-values. As such, it not only mitigates the nonstationary effect without exploding the input of the centralized critic but also safeguards the business confidentiality of FCSs. Moreover, a sequential updating scheme is used to ensure policy monotonic improvement and a Bayesian inference technique is adopted to enhance the robustness of the pricing strategy. Case studies on a large-scale test CTPN system reveal that the proposed method facilitates sufficient competition among FCSs, which is able to drive down the average charging prices for EV users. It also smooths out the spatial distribution of EV charging demands, which reduces the traffic congestions in the TN while enhancing the wind absorption and cost efficiency of the PDN.