Personalized federated learning: A Clustered Distributed Co-Meta-Learning approach

Federated Learning (FL) aims to train a model across multiple parties while preserving the privacy of users' data. Traditional FL only develops a common model for users, and does not adapt the model to each user. Therefore, personalized FL approaches emerged that can further adapt the model to users, thus showing better performance. Among these personalized FL methods, meta-learned personalized FL methods achieve the advanced performance. However, this personalization scheme adapts model to each user according to their own data, and the features it learned are not enough and not rich, especially when there are extremely little data in some users. In this paper, we study a more effective variant of personalization federated learning. We first formalize a new learning problem and propose a Distributed Co-Meta-Learning approach for this learning problem. Then, we show how to design a new personalized FL framework based on this Distributed Co-Meta-Learning approach. To optimize our proposed personalized FL framework, while reducing the computational cost in the optimization, we study a chain-estimation aggregation method for our framework. It also reduces the computational load in the clients. Further, we give the theoretical convergence analysis of our method on the most complex case, non-convex and non-IID problems, and analyze some parameters' properties within it. Experiments demonstrate that our method achieves the state-of-the-art performance in the personalization FL area