Privacy-Preserving Federated Bayesian Optimization with Learnable Noise

Conventional Bayesian optimization approaches assume that all available data are located on one device, which does not consider privacy concerns since data storage and transmission may pose threats to data security. Existing differential privacy-based approaches can protect sensitive information by adding well-calibrated noise to the real objective value of the query input, which may seriously degrade the performance of Bayesian optimization. To address this issue, we propose to learn the noise level of each solution instead of the newly infilled solutions by optimizing a utility-privacy function that considers obfuscating the information of the current best solution, and striking a balance between exploration and exploitation. In this way, the real objective values and the current best solution will be protected. We further extend the proposed approach to a federated setting by considering multiple clients. Our experimental results show that the proposed algorithm can achieve very competitive optimization performance on ten test functions while being able to preserve data privacy. In addition, at the lowest level of privacy protection, the current best solution is leaked in less than 5 out of 91 rounds of surrogate updates for the proposed algorithm, which is significantly smaller than that of the algorithm under comparison.