Efficient Federated Learning Through Distributed Model Pruning
作者
Mohammad Munzurul Islam,Mohammed A. Alawad
出处
期刊:IEEE Computer Society Annual Symposium on VLSI日期:2024-07-01卷期号:33: 155-160被引量:1
标识
DOI:10.1109/isvlsi61997.2024.00038
摘要
Federated Learning (FL) has emerged as a crucial paradigm for training machine learning models across distributed devices while preserving data privacy. However, the resource constraints of these devices pose a significant challenge to training efficiency. Model pruning offers a promising solution by reducing the complexity of neural networks, thereby alleviating the computational burden on resource-constrained devices. Existing pruning approaches, however, struggle to reconcile the computational demands of pruning with the heterogeneity of data across devices in the FL setting. To address these challenges, we propose a novel approach (SparseFL) for efficient FL through distributed model pruning. Our method leverages sparsity regularization and compressive sensing techniques to assess the importance of model parameters, enabling selective pruning to mitigate computational overhead while maintaining model performance. By integrating compressive sensing with FL, we create a distributed pruning framework that adapts to data heterogeneity across devices, improving both computational efficiency and model performance. In our experiments, conducted on standard image classification tasks, our proposed approach demonstrates significant enhancements in terms of computation efficiency and model accuracy compared to baseline methods. Our results underscore the efficacy of distributed model pruning to enhance FL in resource-constrained environments.