Privacy Amplification by Decentralization

Analyzing data owned by several parties while achieving a good trade-off\nbetween utility and privacy is a key challenge in federated learning and\nanalytics. In this work, we introduce a novel relaxation of local differential\nprivacy (LDP) that naturally arises in fully decentralized algorithms, i.e.,\nwhen participants exchange information by communicating along the edges of a\nnetwork graph without central coordinator. This relaxation, that we call\nnetwork DP, captures the fact that users have only a local view of the system.\nTo show the relevance of network DP, we study a decentralized model of\ncomputation where a token performs a walk on the network graph and is updated\nsequentially by the party who receives it. For tasks such as real summation,\nhistogram computation and optimization with gradient descent, we propose simple\nalgorithms on ring and complete topologies. We prove that the privacy-utility\ntrade-offs of our algorithms under network DP significantly improve upon what\nis achievable under LDP, and often match the utility of the trusted curator\nmodel. Our results show for the first time that formal privacy gains can be\nobtained from full decentralization. We also provide experiments to illustrate\nthe improved utility of our approach for decentralized training with stochastic\ngradient descent.\n