FedLMA: A Federated Learning Framework Integrating LLM-Based Multi-Agent Reasoning With Knowledge Distillation

Internet of Medical Things (IoMT) applications encounter issues with data distribution bias, privacy and security concerns, and resource constraints, especially in consumer-centric scenarios. Federated Learning (FL) has emerged as a distributed, privacy-preserving paradigm that enables multiple decentralized clients to collaboratively train a shared model using only their own local data. However, conventional FL methods often suffer from client drift induced by non-IID data across participants, resulting in slower convergence and degraded overall performance. To address these challenges, we propose FedLMA, a fully decentralized framework that combines local knowledge distillation with an LLM-driven multi-agent reasoning mechanism over a blockchain-enabled knowledge-sharing layer. Specifically, FedLMA first extracts and regularizes soft labels from each client’s local model to generate robust distilled knowledge. Then, a large language model driven multi-agent system evaluates class-wise learning difficulty and knowledge retention to form a personalized demand vector, which is used to adaptively filter and weight relevant knowledge fragments. Finally, the selected knowledge guides student model updates via a hybrid cross-entropy and KL-divergence loss. Comprehensive experiments on heterogeneous benchmarks demonstrate that FedLMA significantly outperforms state-of-the-art personalized FL methods in both accuracy and convergence speed under highly non-IID data distributions, effectively mitigating client drift and enhancing overall model performance in decentralized, heterogeneous environments.