差别隐私
计算机科学
协议(科学)
联合学习
频谱分析仪
原始数据
简单(哲学)
机器学习
梯度升压
人口
人工智能
数据挖掘
随机森林
社会学
人口学
电信
病理
程序设计语言
替代医学
医学
哲学
认识论
作者
Ruixuan Liu,Yang Cao,Hong Chen,Ruoyang Guo,Masatoshi Yoshikawa
出处
期刊:Proceedings of the ... AAAI Conference on Artificial Intelligence
[Association for the Advancement of Artificial Intelligence (AAAI)]
日期:2021-05-18
卷期号:35 (10): 8688-8696
被引量:19
标识
DOI:10.1609/aaai.v35i10.17053
摘要
Federated Learning (FL) is a promising machine learning paradigm that enables the analyzer to train a model without collecting users' raw data. To ensure users' privacy, differentially private federated learning has been intensively studied. The existing works are mainly based on the curator model or local model of differential privacy. However, both of them have pros and cons. The curator model allows greater accuracy but requires a trusted analyzer. In the local model where users randomize local data before sending them to the analyzer, a trusted analyzer is not required but the accuracy is limited. In this work, by leveraging the \textit{privacy amplification} effect in the recently proposed shuffle model of differential privacy, we achieve the best of two worlds, i.e., accuracy in the curator model and strong privacy without relying on any trusted party. We first propose an FL framework in the shuffle model and a simple protocol (SS-Simple) extended from existing work. We find that SS-Simple only provides an insufficient privacy amplification effect in FL since the dimension of the model parameter is quite large. To solve this challenge, we propose an enhanced protocol (SS-Double) to increase the privacy amplification effect by subsampling. Furthermore, for boosting the utility when the model size is greater than the user population, we propose an advanced protocol (SS-Topk) with gradient sparsification techniques. We also provide theoretical analysis and numerical evaluations of the privacy amplification of the proposed protocols. Experiments on real-world dataset validate that SS-Topk improves the testing accuracy by 60.7% than the local model based FL. We highlight an observation that SS-Topk improves the accuracy by 33.94\% than the curator model based FL without any trusted party. Compared with non-private FL, our protocol SS-Topk only lose 1.48% accuracy under (2.348, 5e-6)-DP per epoch.
科研通智能强力驱动
Strongly Powered by AbleSci AI