规范化(社会学)
歪斜
概念漂移
计算机科学
原始数据
趋同(经济学)
人工智能
标准差
差异(会计)
机器学习
数据挖掘
统计
数学
电信
业务
会计
数据流挖掘
社会学
人类学
经济
程序设计语言
经济增长
作者
Myeongkyun Kang,Soopil Kim,Kyong Hwan Jin,Ehsan Adeli,Kilian M. Pohl,Sang Hyun Park
标识
DOI:10.1016/j.patcog.2023.110230
摘要
Federated Learning (FL) allows a global model to be trained without sharing private raw data. The major challenge in FL is client-wise data heterogeneity leading to different model convergence speed and accuracy. Despite the recent progress of FL, most methods verify their accuracy on prior probability shift (label distribution skew) dataset, while the concept drift problem (i.e., where each client has distinct styles of input while sharing the same labels) has not been explored. In real scenarios, concept drift is of paramount concern in FL since the client’s data is collected under extremely different conditions making FL optimization more challenging. Significant differences in inputs among clients exacerbate the heterogeneity of clients’ parameters compared to prior probability shift, ultimately resulting in failures for previous FL approaches. To address the challenge of concept drift, we use Weight Normalization (WN) and Adaptive Group Normalization (AGN) to alleviate conflicts during global model updates. WN re-parameterizes weights to have zero mean and unit variance while AGN adaptively selects the optimal mean and standard deviation for feature normalization based on the dataset. These two components significantly contribute to having consistent activations after global model updates reducing heterogeneity in concept drift data. Comprehensive experiments on seven datasets (with concept drift) demonstrate that our method outperforms five state-of-the-art FL methods and shows faster convergence speed compared to the previous FL methods.
科研通智能强力驱动
Strongly Powered by AbleSci AI