Secure Aggregation in Heterogeneous Federated Learning for Digital Ecosystems

Privacy-preserving federated learning (PPFL) is vital for Industry 5.0 digital ecosystems due to the increasing number of interconnected devices and the volume of shared sensitive data. Secure aggregation (SA) protocols are essential components to fulfill the privacy properties of PPFL. However, there are still fundamental challenges to be tackled. For example, statistical and model heterogeneous characteristics across terminal devices, communication bottlenecks, and the requirement of inputting integers rather than real values in cryptographic operations. To overcome these problems, we propose RF-HFL, a novel secure aggregation scheme for PPFL in digital ecosystems with the capability to act on non-independent and identically distributed (non-IID) data. The scheme distills Representative Factors from each edge device through a self-attention-based neural network and transfers the average of these factors to the server for aggregation. The central model is then sent back to the devices for regularizing decentralized training models. The data transmitted between the devices and server are greatly reduced, henceforth significantly decreasing the communication overhead in PPFL. Theoretical analyses are provided for correctness, security and convergence. We set a benchmark for comparing our proposed scheme with several state-of-the-art SA protocols or algorithms in heterogeneous PPFL. Results demonstrate the effectiveness and efficiency of RF-HFL on multiple datasets.