Towards Assessing the Impact of Bayesian Optimization's Own\n Hyperparameters

Bayesian Optimization (BO) is a common approach for hyperparameter\noptimization (HPO) in automated machine learning. Although it is well-accepted\nthat HPO is crucial to obtain well-performing machine learning models, tuning\nBO's own hyperparameters is often neglected. In this paper, we empirically\nstudy the impact of optimizing BO's own hyperparameters and the transferability\nof the found settings using a wide range of benchmarks, including artificial\nfunctions, HPO and HPO combined with neural architecture search. In particular,\nwe show (i) that tuning can improve the any-time performance of different BO\napproaches, that optimized BO settings also perform well (ii) on similar\nproblems and (iii) partially even on problems from other problem families, and\n(iv) which BO hyperparameters are most important.\n