Data-Driven Ranking and Selection Under Input Uncertainty

In many applications, input data are collected frequently to update the simulation model of the system, whereas simulation is run to compare different designs/strategies to identify the best one with a high confidence. In “Data-Driven Ranking and Selection Under Input Uncertainty,” Wu, Wang, and Zhou consider such a simulation-based ranking and selection (R&S) problem, in which the input distribution is estimated and updated with input data arriving in batches over time. Unlike most existing works of R&S that conduct simulation under a fixed distribution, in this data-driven setting, simulation outputs are generated under different input distributions over time. A moving average estimator is introduced to aggregate simulation outputs generated under heterogenous distributions. Then, two sequential elimination procedures are devised by establishing exact and asymptotic confidence bands for the estimator. The efficiency of the procedures can be further boosted by incorporating the “indifference zone” idea and optimizing the “drop rate” parameter of the moving average estimator.