Federated Learning With Lossy Lempel–Ziv Gradient Compression

Federated Learning (FL) has gained widespread adoption for distributed model training due to its ability to preserve privacy and deliver promising learning performance. However, the limited availability of communication resources can significantly slow down the training process. To address this issue, this paper proposes two gradient compression schemes based on Lempel-Ziv (LZ) coding: Lossy Lempel-Ziv with element-wise distortion$p$(LLZ-$p$) and Lossy Lempel-Ziv with Side Information (LLZ-SI). The LLZ-$p$algorithm exploits the inherent redundancy within local gradients and introduces a controlled distortion into the vanilla Lempel-Ziv scheme. Our convergence analysis demonstrates that LLZ-$p$can not only notably reduce communication costs, but also achieve a sublinear convergence rate comparable to that of its lossless counterpart. LLZ-SI further enhances compression by exploiting historical gradient information available at both clients and the server, effectively balancing the compression of current and past gradients. Extensive experiments on various datasets and learning models demonstrate that the proposed LZ-based schemes reduce communication by 58% without degrading accuracy and by up to 90% while maintaining nearly the same learning performance.