Annealed Entropic Allocation for Ranking and Selection

Abstract: We propose Annealed Entropic Allocation, an annealed weighted soft-min framework for sequential budget allocation in ranking and selection. The central idea is to replace the non-smooth maximin large-deviation rate objective with a weighted log-sum-exp surrogate that aggregates challenger-specific pairwise scores through soft-min weights, mitigating hard switching when several challengers are nearly active. To improve finite-budget discrimination, we incorporate the saddlepoint approximation -- a sub-exponential correction derived from refined pairwise tail asymptotics. Because these corrections are sub-exponential and the smoothing parameter is annealed to zero, the surrogate preserves the same first-order large-deviation target as the classical maximin formulation. We show that the surrogate converges uniformly to the hard minimum, that the soft-min weights concentrate on the active challengers, and that, under fixed weights, the induced target allocation map is continuous on the simplex interior. Numerical experiments on Gaussian and exponential instances demonstrate competitive performance, especially when multiple challengers are nearly tied.

• The paper introduces Annealed Entropic Allocation, which replaces the unstable hard minimum over challenger error rates with a Gibbs-weighted soft minimum that converges to the LD-optimal maximin allocation as the budget grows.
• Saddlepoint-based weights provide finite-budget corrections that favor challengers with heavier tails when exponents are nearly tied, while cumulative tracking guarantees realized allocations remain within K−1 samples of target counts.
• Across six Gaussian and exponential test instances, AEA was consistently among the strongest methods, achieved the lowest moderate- and large-budget PICS on one heterogeneous Gaussian case, and ran up to 9× faster than BOLD and 6× faster than TS-KKT-IDS.

Motivation and problem setting

In ranking and selection (R&S), a simulation budget must be allocated sequentially across $K$ alternatives to maximize the probability of correct selection (PCS). The large-deviation (LD) perspective of Glynn and Juneja characterizes the exponential decay rate of the false-selection probability under static allocations and identifies the rate-optimal allocation as the solution of a concave maximin program over pairwise LD rates $C_{b,j}(p)$, where $b$ is the true best design and $j$ ranges over challengers. While this characterization is asymptotically correct, the outer minimum in the maximin objective is nonsmooth: when several challengers have nearly equal pairwise rates, small estimation errors can switch which challenger appears least favorable, producing kinks and potentially oscillatory behavior in sequential allocation rules. Refined finite-budget PCS approximations, such as the Bahadur–Rao expansions underlying FCBA, improve the accuracy of the objective but do not resolve this active-challenger instability.

The paper proposes Annealed Entropic Allocation (AEA), which replaces the hard minimum with an annealed weighted soft-min surrogate. Challenger-specific pairwise scores are aggregated through Gibbs weights derived from a log-sum-exp form, spreading sampling attention across nearly active challengers at finite budgets while sharpening toward the hard maximin focus as the budget grows. A saddlepoint pre-factor from refined pairwise tail asymptotics supplies a principled finite-budget tie-breaker among near-active challengers.

The annealed soft-min surrogate

For annealing level $t$ (an inverse-temperature parameter), the surrogate is

$$F_t(p) = -\frac{1}{t}\log\Big(\sum_{j\neq b}\rho_{j,t}\exp(-t\,C_{b,j}(p))\Big),$$

where $\rho_{j,t}>0$ are challenger weights acting as lower-order corrections $-t^{-1}\log\rho_{j,t}$ to the pairwise exponents. Provided the weights are subexponential, $\max_j|\log\rho_{j,t}|=o(t)$, the surrogate converges uniformly to the hard minimum:

$$\sup_{p\in\Delta^{K-1}}|F_t(p)-m(p)| \le \frac{L_t+\log(K-1)}{t}\to 0,$$

with $C_{b,j}(p)$0 and $C_{b,j}(p)$1. Under continuity of the pairwise rates and uniqueness of the maximin optimizer $C_{b,j}(p)$2, any sequence of maximizers of $C_{b,j}(p)$3 converges to $C_{b,j}(p)$4. Thus the smoothing and the weight corrections affect only the finite-budget path; they leave the first-order LD target intact.

Because each pairwise rate is concave and continuously differentiable on the simplex interior (the non-smoothness resides solely in the outer minimum), $C_{b,j}(p)$5 inherits concavity and differentiability for fixed weights. Its gradient is a convex combination of pairwise gradients with coefficients given by the Gibbs weights

$C_{b,j}(p)$6

These weights shift continuously across challengers rather than switching abruptly, which is the key stabilization mechanism.

Target allocation map and tracking

Using the envelope theorem, the marginal contribution of alternative $C_{b,j}(p)$7 against challenger $C_{b,j}(p)$8 equals its KL divergence contribution $C_{b,j}(p)$9 evaluated at the Chernoff projection. This yields challenger-specific allocation vectors $b$0, and the weighted target allocation $b$1 can be written in two equivalent ways: as a Gibbs-weighted aggregation of the $b$2, or as the normalized vector of current shares times marginal surrogate gains, $b$3. When one challenger dominates the Gibbs mass, $b$4 approaches that challenger's pairwise allocation; when several are near-active, it averages them smoothly.

Two structural results follow. First, with fixed positive weights and continuously differentiable pairwise rates, $b$5 is continuous on the interior of the simplex, and Lipschitz on compact subsets where the rates have bounded Hessian—formalizing the contrast with hard-minimum rules that jump discontinuously when the worst-case challenger changes. Second, the Gibbs weights concentrate on the active set $b$6: for any non-active $b$7, the ratio $b$8 decays exponentially because the subexponential weight ratio cannot offset the exponent gap. The sequential policy implements these targets via cumulative tracking, for which the paper proves the uniform bound $b$9, so realized proportions converge to the Cesàro average of the targets whenever that average converges.

Saddlepoint weights

The remaining design choice—the weights—is resolved via the saddlepoint approximation to each pairwise error probability. For a sum-scale statistic $j$0 whose left tail defines the error event, the approximation gives

$j$1

where $j$2 solves $j$3. Challenger-independent factors cancel in the Gibbs normalization, leaving effective weights $j$4. These are order $j$5, hence subexponential, so the first-order target is unchanged. Their role is a finite-budget tie-breaker: when two challengers have nearly equal exponents, the exponential factor in the Gibbs ratio is close to one, and the challenger with the heavier tail at the same exponent receives more sampling effort. For Gaussian alternatives this recovers the Mills-ratio pre-factor; for exponential alternatives it yields a closed form through the CGF saddlepoint.

One caveat deserves emphasis: because the saddlepoint weights depend on the allocation, the algorithm adopts a fixed-weight convention, evaluating them at the current empirical allocation before computing the target. Concavity and the gradient formula hold exactly only under this frozen-weight convention; the theoretical guarantees for $j$6 likewise assume fixed weights, so the analysis does not fully cover the joint dynamics of allocation-dependent weighting.

Numerical evidence

Experiments use four Gaussian instances ($j$7 with three mean/variance configurations plus a $j$8 instance) and two exponential instances, with $j$9 macro-replications per instance and $t$0 initial replications. Benchmarks include BOLD, OEA, FCBA, TS-KKT-IDS, TTTS, standard Thompson sampling, OCBA-MSA, and equal allocation.

The central empirical finding is that AEA is the only method consistently in the leading group across all six instances, whereas every baseline is markedly instance-dependent: TS-KKT-IDS performs essentially as well as AEA on G1 and G4 but degrades sharply on G3; OCBA-MSA is competitive on G1 but among the weakest on G2 and G4; TTTS is strongest on G3 and at small budgets but loses ground later; BOLD deteriorates sharply on E2. The advantage is most pronounced on symmetric instances (G2, E2) where no single challenger dominates—precisely the regime the soft-min aggregation targets. On G3, AEA attains the lowest PICS at moderate and large budgets despite heterogeneous variances.

Computationally, AEA's closed-form updates make it fast: on G3 it is roughly $t$1 faster than BOLD and $t$2 faster than TS-KKT-IDS; on the exponential instances it runs at 11–12 ms per macro-replication versus 57–90 ms for BOLD and 138–367 ms for TTTS. OCBA-MSA is fastest on Gaussian instances but does not extend to exponential sampling.

Limitations and open questions

The theoretical analysis is first-order: it establishes preservation of the LD-optimal ratios and qualitative properties of the surrogate, but provides no finite-sample PCS guarantee or regret bound for the sequential policy, and the interaction between the annealing schedule and the saddlepoint weights is not analyzed jointly. The fixed-weight convention sidesteps, rather than resolves, the dependence of the weights on the allocation. The framework assumes independent samples from regular one-parameter exponential families, excluding correlated designs, common random numbers, covariates, continuous design spaces, and input uncertainty—all identified by the authors as open extensions. Adaptation to probability-of-good-selection criteria also remains unaddressed.

Conclusion

AEA offers a smooth, annealed surrogate for the LD maximin objective in R&S, combining log-sum-exp aggregation of challenger scores with saddlepoint-based finite-budget corrections. The theory establishes uniform convergence to the hard minimum, concentration of Gibbs weights on the active set, and continuity of the induced allocation map, while experiments show consistent empirical performance across Gaussian and exponential instances—particularly where multiple challengers are nearly tied—at competitive computational cost.