Stochastic Gradient Tracking Methods for Distributed Personalized Optimization over Networks

In this paper, we consider a distributed optimization problem over a network of nodes whose cost functions depend on a decision variable consisting of two parts: global (shared) part and local (node-specific) part. This problem structure arises in several important scenarios where the global part captures the common feature among nodes and the local part represents the personalized feature. To solve this problem, we develop a new personalized distributed stochastic gradient tracking method, where each node locally updates variables in a stochastic way and communicate the shared part with its neighbors for coordination. Leveraging a proper Lyapunov design, we show that the proposed algorithm converges linearly to a neighborhood of the optimum for smooth and strongly convex objective functions. The obtained rate result shows a clear dependence of the convergence performance on the topology and the properties of the objective functions. Numerical examples illustrate the effectiveness of the proposed algorithm towards mitigating data heterogeneity among nodes.