Analyzing and provably improving fixed budget ranking and selection algorithms

Abstract: This paper studies the fixed budget formulation of the Ranking and Selection (R&S) problem with independent normal samples, where the goal is to investigate different algorithms' convergence rate in terms of their resulting probability of false selection (PFS). First, we reveal that for the well-known Optimal Computing Budget Allocation (OCBA) algorithm and its two variants, a constant initial sample size (independent of the total budget) only amounts to a sub-exponential (or even polynomial) convergence rate. After that, a modification is proposed to achieve an exponential convergence rate, where the improvement is shown by a finite-sample bound on the PFS as well as numerical results. Finally, we focus on a more tractable two-design case and explicitly characterize the large deviations rate of PFS for some simplified algorithms. Our analysis not only develops insights into the algorithms' properties, but also highlights several useful techniques for analyzing the convergence rate of fixed budget R&S algorithms.