Behavioral Study of Bayesian Neural Networks Under a Typical Surrogate Model-Assisted Evolutionary Search Framework

The machine learning method for surrogate modeling is a keystone in surrogate model-assisted evolutionary algorithms (SAEAs). The current arguably most widely used surrogate modeling methods in SAEAs are Gaussian process and radial basis function. This paper investigates the behavior of a machine learning alternative, Bayesian neural network (BNN), which has seldom been paid attention to in SAEAs. Through empirical methods, the properties of the BNN model, its co-work with prescreening methods, and its comparison with other machine learning alternatives are investigated using a typical SAEA model management framework. Experimental results show the behavior of BNN models in terms of surrogate model prediction accuracy, the availability of prediction uncertainty estimation, and the training cost, demonstrating the potential of BNN being a competitive surrogate alternative in SAEAs. This is validated in various experiments, in which, under the same SAEA model management framework, SAEAs using BNN as the surrogate model save 83.1% and 42.5% modeling time compared to those using GP and the drop-out method, respectively.