ROCFL: A Robust Clustered Federated Learning Framework towards Heterogeneous Data

As a decentralized machine learning approach, federated learning has become a key solution for numerous applications. However, one primary challenge lies in the data heterogeneity issue when developing effective federated learning algorithms. While many existing frameworks address the concern of non-independent and identically distributed data(non-IID), they have not considered heterogeneity of data quality. Therefore, we propose ROCFL, a robust clustered federated learning method in this study, which can amplify the disparity in weight allocation between models trained on different quality data. We first develop an optimal clustering matching mechanism that groups clients with similar data distributions. This allows the optimal clustering model to be derived without specifying a predetermined number of clusters. Next, we introduce a personalized weight allocation strategy, which assigns a weight benchmark to each cluster based on its cluster importance index. This strategy mitigates the negative impacts of low-quality data during model aggregation. Finally, we design a federated aggregation strategy grounded in a sampling approach. It not only ensures unbiased sampling but also significantly reduces both computational and communication overheads. Experiments have been carried out and the results demonstrate that higher accuracy and robustness can be achieved by ROCFL in scenarios wherein data distribution and quality heterogeneity coexist, in comparison with the benchmarks.