One-Shot Screening of Simulated Systems for Acceptability

Abstract: We introduce a general-purpose framework for designing screening procedures for problems featuring a finite set of simulated systems, a.k.a. ranking-and-selection problems. The framework offers a novel perspective on screening in which decisions to retain or eliminate systems are based on confidence regions for the unknown problem instance rather than comparisons of estimated performances. Specifically, a system is retained if it has acceptable performance under some plausible configuration of response vectors contained in the confidence region. This perspective facilitates the design of procedures that guarantee to return either all acceptable systems, or each acceptable system, with high probability and accommodates many well-studied definitions of acceptability, including feasibility with respect to stochastic constraints and optimality with respect to one or more objectives. We further study a subclass of the framework that yields simple and computationally efficient screening procedures that often have lower-order time complexity than existing methods and naturally supports parallelization without loss of screening power. We demonstrate the effectiveness and efficiency of the procedures through numerical experiments.