Federated Chain Context Optimization for Long-Tailed Multi-Label Image Classification

Federated learning is an emerging machine learning paradigm that effectively alleviates the data silo problem by distributing the model training process to multiple data holders. However, data from real-world mobile applications often has multi-label and presents a long-tailed distribution, where labels are generally non-independent and non-identically distributed, thereby increasing the challenges caused by data heterogeneity. To address the above problems, we propose a Federated Chain Context Optimization (FedCCO) for long-tailed multi-label image classification. Inspired by the success of Chain of Though (CoT) in enhancing the semantic expressive ability of models, this method fine-tunes the CLIP model using semantic descriptive vectors generated by the Chain Context Optimization (ChCoOp) to establish semantic correlations between head and tail classes across clients, which improves the ability of the model to recognize tail classes. The experimental results show that the FedCCO achieves satisfactory performance in long-tailed multi-label image classification in federated learning on VOC-LT and COCO-LT datasets.