Dynamic Optimization of Edge Aggregation Structures and Update Frequencies for Efficient Distributed Hierarchical Model Training

Edge computing enables distributed machine learning models to be deployed and trained near the user space. However, the intricate nature of edge computing raises several challenges to distributed machine learning frameworks: 1) inferior convergence arising from non-independent and identically distributed (non-IID) edge data; 2) inefficient structural adaptation, where device dynamism complicates the adjustment of aggregation structure; and 3) reduced training efficiency, as resource heterogeneity and fluctuations create systemic stragglers. To address these issues, a distributed hierarchical model training framework has been proposed by considering the dynamic aggregation structure and frequency in this paper. This framework designs an Edge Aggregation Structure and Frequency method, namely EASF, for distributed model training in heterogeneous edge computing environments. First, a dynamic distributed aggregation structure method is formulated to consider various data distribution patterns. This method constructs and modifies the aggregation structure in a distributed manner to adapt to variations in working edge devices. Second, a self-adapted aggregation frequency method and a timeout abandonment mechanism are proposed to allow each node to update its aggregation frequency adaptively. Lastly, a theoretical analysis demonstrates the convergence property of the EASF method in dynamic environments. Extensive experiments have been conducted on a set of open testbeds. Results show that the EASF significantly improves the efficiency and accuracy of hierarchical model training in heterogeneous edge computing.