Efficient hyperparameter tuning for kernel ridge regression with\n Bayesian optimization

Machine learning methods usually depend on internal parameters -- so called\nhyperparameters -- that need to be optimized for best performance. Such\noptimization poses a burden on machine learning practitioners, requiring expert\nknowledge, intuition or computationally demanding brute-force parameter\nsearches. We here address the need for more efficient, automated hyperparameter\nselection with Bayesian optimization. We apply this technique to the kernel\nridge regression machine learning method for two different descriptors for the\natomic structure of organic molecules, one of which introduces its own set of\nhyperparameters to the method. We identify optimal hyperparameter\nconfigurations and infer entire prediction error landscapes in hyperparameter\nspace, that serve as visual guides for the hyperparameter dependence. We\nfurther demonstrate that for an increasing number of hyperparameters, Bayesian\noptimization becomes significantly more efficient in computational time than an\nexhaustive grid search -- the current default standard hyperparameter search\nmethod -- while delivering an equivalent or even better accuracy.\n