计算机科学
块(置换群论)
正规化(语言学)
人工智能
启发式
机器学习
人工神经网络
理论计算机科学
数学
几何学
作者
Jianchun Liu,Qingmin Zeng,Hongli Xu,Hongli Xu,Zhiyuan Wang,He Huang
出处
期刊:IEEE ACM Transactions on Networking
[Institute of Electrical and Electronics Engineers]
日期:2023-01-01
卷期号:: 1-15
标识
DOI:10.1109/tnet.2023.3301972
摘要
Federated Learning (FL) is a distributed model training framework that allows multiple clients to collaborate on training a global model without disclosing their local data in edge computing (EC) environments. However, FL usually faces statistical heterogeneity (e.g., non-IID data) and system heterogeneity (e.g., computing and communication capabilities), resulting in poor model training performance. To deal with the above two challenges, we propose an efficient FL framework, named FedBR , which integrates the idea of block-wise regularization and knowledge distillation (KD) into the pioneering FL algorithm FedAvg , for resource-constrained edge computing. Specifically, we first divide the model into multiple blocks according to the layer order of deep neural network (DNN). The server only sends some consecutive model blocks instead of an entire model to clients for communication efficiency. Then, the clients make use of knowledge distillation to absorb the knowledge of global model blocks to alleviate statistical heterogeneity during local training. We provide a theoretical convergence guarantee for FedBR and show that the convergence bound will decrease as the increasing number of model blocks sent by the server. Besides, since the increasing number of model blocks brings more computing and communication costs, we design a heuristic algorithm (GMBS) to determine the appropriate number of model blocks for clients according to their varied data distributions, computing, and communication capabilities. Extensive experimental results show that FedBR can reduce the bandwidth consumption by about 31%, and achieve an average accuracy improvement of around 5.6% compared with the baselines under heterogeneous settings.
科研通智能强力驱动
Strongly Powered by AbleSci AI