Simulated Annealing, Weighted Simulated Annealing and Genetic Algorithm at Work

Two well known stochastic optimization algorithms, simulated annealing and genetic algorithm are compared when using a sample to minimize an objective function which is the expectation of a random variable. Since they lead to minimum depending on the sample, a weighted version of simulated annealing is proposed in order to reduce this kind of over-fit bias. The algorithms are implemented on an optimization problem related to quality control. A design of experiment is used to get the best trade-off between optimization and execution time. Simulated annealing appears to be more efficient than the genetic algorithm. With regard to the bias problem, the randomly weighted version of simulated annealing allows to achieve a solution less dependent on the sample and thus less biased.