Future Trajectory Representation-Aided DRL for Real-Time Battery Energy Storage Dispatch in Distribution Networks

Dispatching battery energy storage systems (BESS) in distributed grids is challenging due to the sequential nature of the decision-making process, which needs to account for future uncertainties in system states and time-coupling constraints. Numerous forecasting techniques have been developed to capture such uncertainties, but most prioritize accuracy alone, and their potential for enhancing optimization performance has rarely been studied. To bridge this gap, this paper proposes a novel forecasting-aided deep reinforcement learning (DRL)-based BESS dispatch approach that incorporates future trajectory representation into the DRL state space to facilitate more proactive decisions. Specifically, an error-free representation network is first trained offline using self-supervised contrastive learning to encode upcoming electricity prices, load demand, and renewable generation trajectories. It explicitly disentangles trend and seasonal patterns using a mixture of autoregressive experts and a learnable Fourier layer. During online control, the DRL agent receives both the current system state and the learned forecast representation, which are fused using a dynamic routing attention mechanism that adaptively balances present versus future information. The trajectory contrastive learning allows the proposed method to better capture the temporal dynamics of stochastic variables and generate representations that are easier for the agent to exploit. By incorporating structured predictive representations into the control loop, the proposed method can make more forward-looking charging and discharging decisions that consider future uncertainties. Extensive simulations on the IEEE 34-node and 123-node test systems demonstrate that the proposed method yields more cost-efficient BESS control policies compared to baseline approaches.