Visually Analysing the Fairness of Clustered Federated Learning with Non-IID Data

As a popular distributed privacy-preserving machine learning approach, federated learning (FL) trains a global model across numerous clients without directly sharing their private training data. A fundamental challenge of FL is non-independently and identically distributed (non-IID) data, which can be addressed by clustered FL (CFL) that explores inherent partitions among clients to capture heterogeneous local data distributions. Existing CFL methods, however, tend to optimise the mean accuracy across all clients, overlooking the important issue of fairness, that is, uniformity of clients' performance. Besides, prior FL- related visual analytics (VA) frameworks are developed based on the standard FL scenario of single global model and are hence inapplicable to the CFL case of multiple global (cluster) models. To fill the research gap, this study proposes a novel VA framework named cCFLvis to illustrate and analyse variation of fairness during the CFL learning process. A new metric, change of absolute deviation from the mean (ΔAD), is introduced to quantify such variation. Client embeddings are generated by contrastive learning-based dimensionality reduction techniques to facilitate 2-d visualisation of client disparity. Visual cues can be drawn to help understand fairness variation and to offer clues for improving learning outcomes. cCFL vis is a general framework applicable to various CFL methods. Its effectiveness is showcased in a simple demonstration and two case studies, each conducted with a distinct representative CFL method and a different federated non-IID dataset.