Dynamic Sample Selection for Federated Learning with Heterogeneous Data in Fog Computing

Federated learning is a state-of-the-art technology used in the fog computing, which allows distributed learning to train cross-device data while achieving efficient performance. Many current works have optimized the federated learning algorithm in homogeneous networks. However, in the actual application scenario of distributed learning, data is independently generated by each device, and this non-homologous data has different distribution characteristics. Therefore, the data used by each device for local learning is unbalanced and non-IID, and the heterogeneity of data affects the performance of federated learning and slows down the convergence. In this paper, we present a dynamic sample selection optimization algorithm, FedSS, to tackle heterogeneous data in federated learning. FedSS dynamically selects the training sample size during the gradient iteration based on the locally available data size, to settle the expensive evaluations of the local objective function with a massive amount of dataset. We theoretically analyze the convergence and present the complexity estimates of our framework when learning large data from unbalanced distribution. Our experimental results show that the use of dynamic sampling methods can effectively improve the convergence speed with heterogeneous data, and keep computational costs low while achieving the desired accuracy.