A unified data-driven approach under deep reinforcement learning with direct control responses for microgrid operations

Microgrid systems have now seen many integrations with energy storage systems (ESS) and renewable energy sources (RES) to supply cleaner and cheaper energy. A pressing challenge is how to optimally meet the requirements ranging from reducing operational costs and carbon footprints to relieving the grid constraints, alongside the consideration of uncertainties in the supply and demand. To embark on this challenge, this paper proposes a deep reinforcement learning (DRL) approach with direct control responses to optimize multi-objective microgrid operations. First, a new objective function is derived to build a direct response between the control action and the optimization objectives, aiming to improve the learning efficiency. Second, a unified control scheme is designed to study the combined use of past observations and predicted data for microgrid controls. Third, a realistic microgrid model is created to incorporate battery charging and discharging processes with dynamic efficiency and nonlinear battery degradation. Finally, the effectiveness of the proposed approach is validated through various simulations conducted on a US case study, with an additional Norwegian microgrid presented in the supplementary material. The results suggest that the annual reward in the US microgrid can be improved by 139.33% over the baseline (vanilla DQN with a conventional scheme) under perfect predictions, and by 125.45% under noisy predictions. • A direct-response reward function is derived to improve the learning efficiency. • A unified control scheme is designed to organize the state space. • A realistic microgrid model is created for power optimization.