Review of Surrogate Model Algorithms

Surrogate model based optimization algorithms consist in general of the following steps:

In Step 1, an initial experimental design is created and the computationally expensive objective function is evaluated at the selected points. In general, any initial design strategy may be used, but it has to be ensured that there are sufficiently many points to fit the chosen surrogate model in Step 2. The objective function value predictions of the surrogate model at unsampled points are used in Step 3 to select the next evaluation point. After the new function value has been obtained in Step 4, the surrogate model is updated in Step 6 if the stopping criterion has not been satisfied (for example, the budget of function evaluations has not been exhausted) and a new point is selected for evaluation. Otherwise the algorithm stops and returns the best solution found during the optimization in Step 8.

This framework has been adopted in several well-known algorithms for continuous optimization such as EGO (Jones et al, 1998), Gutmann’s RBF method (Gutmann, 2001), DYCORS (Regis and Shoemaker, 2013), SRBF (Regis and Shoemaker, 2007), and SO-M-s (Mu¨ller and Shoemaker, 2014). The major differences between these algorithms are

– the type of surrogate model used to approximate the expensive objective function in Step 2;

– the method for selecting a new evaluation point in Step 3.