Personalized Federated Continual Learning via Multi-Granularity Prompt

Personalized Federated Continual Learning (PFCL) is a new practical scenario\nthat poses greater challenges in sharing and personalizing knowledge. PFCL not\nonly relies on knowledge fusion for server aggregation at the global\nspatial-temporal perspective but also needs model improvement for each client\naccording to the local requirements. Existing methods, whether in Personalized\nFederated Learning (PFL) or Federated Continual Learning (FCL), have overlooked\nthe multi-granularity representation of knowledge, which can be utilized to\novercome Spatial-Temporal Catastrophic Forgetting (STCF) and adopt generalized\nknowledge to itself by coarse-to-fine human cognitive mechanisms. Moreover, it\nallows more effectively to personalized shared knowledge, thus serving its own\npurpose. To this end, we propose a novel concept called multi-granularity\nprompt, i.e., coarse-grained global prompt acquired through the common model\nlearning process, and fine-grained local prompt used to personalize the\ngeneralized representation. The former focuses on efficiently transferring\nshared global knowledge without spatial forgetting, and the latter emphasizes\nspecific learning of personalized local knowledge to overcome temporal\nforgetting. In addition, we design a selective prompt fusion mechanism for\naggregating knowledge of global prompts distilled from different clients. By\nthe exclusive fusion of coarse-grained knowledge, we achieve the transmission\nand refinement of common knowledge among clients, further enhancing the\nperformance of personalization. Extensive experiments demonstrate the\neffectiveness of the proposed method in addressing STCF as well as improving\npersonalized performance. Our code now is available at\nhttps://github.com/SkyOfBeginning/FedMGP.\n