FedRL: Federated Learning with Non-IID Data via Review Learning

Federated Learning epitomizes a sophisticated distributed machine learning methodology, enabling collaborative neural network model training across multiple entities without necessitating the transfer of local data, thereby fortifying data privacy protection. A significant challenge in federated learning lies in the statistical heterogeneity, characterized by non-independent and identically distributed (Non-IID) local data across diverse parties. This heterogeneity can engender inconsistent optimization within individual local models. Although previous research has endeavored to tackle issues stemming from heterogeneous data, our findings indicate that these attempts have not yielded high-performance neural network models. To confront this fundamental challenge, we introduce the FedRL framework in this paper, which facilitates efficient federated learning through review learning. The core principle of FedRL involves leveraging the knowledge representation generated by the global and local model layers to conduct periodic layer-by-layer comparative learning in a reciprocal manner. This strategy rectifies local model training, leading to enhanced outcomes. Our experimental results and subsequent analysis substantiate that FedRL effectively augments model accuracy in image classification tasks, while demonstrating resilience to statistical heterogeneity across all participating entities.