计算机科学
云计算
分布式计算
嵌入
同步(交流)
架空(工程)
联合学习
特征(语言学)
数据共享
数据同步
代表(政治)
数据挖掘
过程(计算)
人工智能
机器学习
计算机网络
医学
频道(广播)
语言学
哲学
替代医学
病理
政治
政治学
法学
操作系统
作者
Yunlong He,Dandan Yan,Fei Chen
标识
DOI:10.1016/j.engappai.2023.106148
摘要
Federated learning can synergize the local model training with private data samples from geo-distributed users. Nevertheless, the unification process of a comprehensive global model through periodical parameter sharing can be time-consuming at a high cost. On the one hand, the data samples are collected from users with diverse preferences, and the data distribution can be non-independent and identically distributed (non-IID). On the other hand, the consequent model trained locally needs to communicate with a remote parameter server periodically for parameter synchronization, which leads to overwhelming communication and synchronization overhead given heterogeneous device capacities and network conditions of end-users. Generally, a hierarchical system design with a clustered group is ideal for accommodating diversity. Actually, it is still challenging to maintain the relationship of the local model training without knowing the data samples in advance for privacy concerns. Therefore, we present hierarchical federated model embedding to formulate the relationship between local data distributions. Initially, the local models are embedded through the global shared dataset to obtain feature latent representation vectors. The cloud server groups the clients according to the vectors, making clients with similar data distribution train collaboratively in a same group. Then, these vectors are used to train the predictor on the cloud server, which is utilized for efficient group assignment when new clients join the system. Compared with the baseline, the accuracy of the group model can be improved by 1.22%∼5.63% and that of the global model can be improved by 3.97%∼14.25% on different datasets.
科研通智能强力驱动
Strongly Powered by AbleSci AI