Personal Federated Learning via Momentum Target with Self-Improvement

Federated learning (FL) is a new artificial intelligence concept that enables Internet-of-Things (IoT) devices to learn a collaborative model without sending the raw data to centralized nodes for processing. Despite numerous advantages, Federated Learning faces huge challenges from model overfitting due to the lack of data and statistical diversity among clients. To address these challenges, we propose a novel personalized federated learning method via momentum adaptation, the so-called pFLTI. Specifically, pFLTI generates the target model by adapting from the temporal ensemble of the meta-learner, i.e., the momentum network. This momentum network and its task-specific adaptations enjoy a favorable generalization performance, enabling self-improving of the meta-learner through knowlodels to find the across task relationsedge distillation. Moreover, we found that perturbing parameters of the meta-learner, e.g., dropout, further stabilize this self-improving process by preventing fast convergence of the distillation loss during meta-training. Our experimental results demonstrate that our algorithm is awesome.