计算机科学
变压器
期限(时间)
人工智能
机器学习
短时记忆
实时计算
数据挖掘
人工神经网络
循环神经网络
电气工程
量子力学
物理
工程类
电压
作者
Siyue Shuai,Xiangjie Kong,Lutong Liu,Wenhong Zhao,Guojiang Shen,Ivan Lee
摘要
The Transformer model, with its ability to capture long-term dependencies, has demonstrated significant potential in enhancing long-term traffic flow prediction for effective urban transportation management. However, most existing Transformer-based methods adhere to a centralized approach, failing to address privacy concerns and to optimize computational resource utilization. Although emerging federated learning paradigms offer privacy protection, the average aggregation still overlooks client heterogeneity and lacks the synchronous efficiency required for traffic flow prediction tasks. Consequently, we introduce FedTFormer, a hierarchical federated learning framework tailored to boost the performance of Transformer models in decentralized environments, involving clients, edge servers, and a central server. Initially, clients are organized into clusters through a sophisticated static clustering mechanism anchored in bipartite graph theory. FedTFormer enhances robustness of Transformer by facilitating synchronous average aggregation within clusters. Additionally, it performs asynchronous fine-tuning of cluster-specific parameters, leveraging hypernetwork constructed on the central server. Clients utilize an optimized Transformer model for localized training, harnessing its proficiency in capturing long-term spatio-temporal dependencies. Ultimately, we conduct extensive experiments across three datasets, comparing our method against ten sophisticated approaches and demonstrating the effectiveness and robustness of FedTFormer.
科研通智能强力驱动
Strongly Powered by AbleSci AI