Advancing Accuracy in Energy Forecasting using Mixture-of-Experts and Federated Learning

Accurate forecasting of load, photovoltaic (PV), and prosumption is essential for energy systems to reliably plan and operate smart grids, improve energy storage optimization, or enable demand response programs. While federated learning could improve prediction accuracy and data privacy by training models locally on private data and sharing only the model parameters, this requires models with enhanced adaptability and generalizability. Therefore, Mixture-of-Experts (MoE) emerges as a promising solution. An MoE layer combines the predictions of multiple specialized sub-models, known as experts, using a gating mechanism to dynamically select and weight the experts' outputs based on the input sequences. Existing deep learning models can integrate this generic layer into their architecture to learn and handle complex patterns in data adaptively. In this paper, we adopt the MoE concept to dense and bidirectional long short-term memory models for load, PV, and prosumption forecasting, using soft- and top-k gating with a custom loss function. As our benchmark, we consider state-of-the-art bidirectional long short-term memory models, convolutional neural networks, residual neural networks, and transformer models. Further, we implement local and federated learning architectures for both static datasets and dynamic data streams. Utilizing the Ausgrid dataset, we demonstrate that including an MoE layer into existing model architectures can significantly increase accuracy. Specifically, we improve the forecasting accuracy for load, PV, and prosumption predictions by 6-9 % (static datasets), 4-8 % (dynamic data streams), and 11-41 % (zero-shot forecasting) while reducing training time by up to 18 %. Since the accuracy of our benchmark models mostly decreases in federated learning due to data heterogeneity, our MoE models consistently show the highest accuracy here. Thus, we demonstrate that federated learning becomes applicable to heterogeneous energy data through our MoE architecture.