Best Arm Identification in Bandits

• Best Arm Identification (BAI) is a pure-exploration problem in multi-armed bandits that seeks to pinpoint the arm with the highest mean reward, distinguishing it from cumulative regret minimization.
• Key methodologies include fixed-confidence and fixed-budget formulations, elimination-based and likelihood-ratio algorithms, and extensions to structured, contextual, transfer, and privacy-constrained settings.
• The literature highlights instance-dependent hardness measures and lower bounds that guide algorithm design, balancing sample complexity and error probability across diverse applications.

Searching arXiv for recent and foundational papers on best arm identification to ground the article in cited work. Best Arm Identification (BAI) is a pure-exploration problem in multi-armed bandits whose objective is to identify the arm with the largest mean reward rather than to maximize cumulative reward. Two classical formulations dominate the literature. In the fixed-confidence setting, the learner stops at a random time and must recommend the unique best arm with probability at least $1-\delta$, while minimizing sample complexity. In the fixed-budget setting, the learner uses exactly $B$ or $T$ samples and seeks to minimize the probability of error or, in some analyses, the simple regret of the final recommendation (Kocák et al., 2020, Azizi et al., 2021, Hwang et al., 20 Feb 2026). Recent work extends BAI beyond unstructured stochastic bandits to structured, contextual, transfer, nonstationary, privacy-constrained, corruption-robust, and Bayesian settings, while also revisiting the relation between BAI and regret minimization (Azize et al., 2024, Shahverdikondori et al., 5 Feb 2025, Hou et al., 2024, Yang et al., 2024).

1. Core formulations

In the standard stochastic setting, there are $K$ arms with unknown means $\mu_1,\dots,\mu_K$, and the best arm is assumed unique. A common normalization is

$$\mu_1>\mu_2\ge \cdots \ge \mu_K, \qquad \Delta_i := \mu_1-\mu_i.$$

In fixed-confidence BAI, the learner adaptively chooses arms $A_t$, stops at a finite time $\tau$, outputs a recommendation, and must satisfy

$$\mathbb{P}_\mu\!\left(A_{\tau+1}\in a^*(\mu)\right)\ge 1-\delta$$

while minimizing $\mathbb{E}_\mu[\tau]$ among all $B$0-correct strategies (Kocák et al., 2020). In fixed-budget BAI, the learner must return an arm after exactly $B$1 pulls and minimize the probability of error

$B$2

or equivalently maximize the probability of identifying the optimal arm after the prescribed budget (Azizi et al., 2021).

Several adjacent objectives are studied under the BAI umbrella. Fixed-budget work often evaluates the final recommendation through the simple regret

$B$3

which measures the loss of the returned arm relative to the true best arm rather than cumulative regret over time (Hwang et al., 20 Feb 2026). Fixed-confidence variants include $B$4-good arm identification, where the target set is

$B$5

and feasible or constrained BAI, where an arm is admissible only if all constraint means lie below a threshold and the goal is to identify the feasible arm with the highest performance (Le et al., 2023, Cai et al., 12 Nov 2025).

A distinct formulation, BAI with minimal regret, makes the tension with regret minimization explicit. Here the learner is required to be $B$6-PAC but, among such algorithms, minimize the expected cumulative regret up to the stopping time: $B$7 This differs from both standard fixed-confidence BAI, which optimizes $B$8, and standard regret minimization, which optimizes regret over a fixed horizon without a stopping-and-recommendation requirement (Yang et al., 2024).

2. Complexity characterizations and lower bounds

A central theme in BAI is the identification of instance-dependent hardness measures. In fixed-confidence structured settings, lower bounds are typically written through a max–min characteristic time. For spectral bandits with graph smoothness, any $B$9-correct strategy satisfies

$T$0

where

$T$1

The smoothness constraint changes the alternative set and therefore the optimal allocation itself (Kocák et al., 2020). An analogous characteristic-time principle appears in BAI with post-action context, where

$T$2

and the inner optimization depends on whether the context is non-separator or separator (Shahverdikondori et al., 5 Feb 2025).

For two-armed Gaussian fixed-budget BAI with known variances, the large-deviation lower bound specialized from Kaufmann et al. yields

$T$3

This exponent highlights the gap $T$4 and the sum of standard deviations $T$5 as the decisive difficulty parameters (Kato, 2023).

Lower bounds also reveal structural tensions between objectives. In BAI with minimal regret, every $T$6-PAC algorithm must satisfy

$T$7

Moreover, if an algorithm is asymptotically optimal for this minimal-regret objective, then

$T$8

so optimal regret and optimal fixed-confidence sample complexity are incompatible (Yang et al., 2024).

The relation between fixed-budget and fixed-confidence BAI has also been clarified at a generic level. A black-box reduction, FC2FB, shows that fixed-budget BAI is no harder than fixed-confidence BAI up to logarithmic factors: if a strong fixed-confidence algorithm has high-probability sample complexity

$T$9

then FC2FB yields a fixed-budget algorithm whose complexity matches this up to logarithmic overheads (Balagopalan et al., 3 Feb 2026). This suggests that characteristic-time analyses developed for fixed-confidence problems can often be transferred algorithmically to fixed-budget settings.

3. Algorithmic paradigms

A large fraction of fixed-budget BAI algorithms are elimination-based. In structured bandits, Generalized Successive Elimination (GSE) operates in

$K$0

stages, splits the budget evenly with stage budget

$K$1

and in each stage projects the active set into its spanned subspace, explores according to an allocation, estimates means under a joint model, and keeps the top $K$2 fraction of arms (Azizi et al., 2021). In corrupted bandits, Probabilistic Sequential Shrinking $K$3 replaces deterministic schedules by uniform random sampling over the active set, which is essential because the adversary can no longer predict the exact arm to be sampled next (Zhong et al., 2020). For anytime BAI under insufficient sampling, Box Thirding (B3) uses ternary comparisons with lift, shift, and discard actions; although each local operation is ternary, the paper notes that the effective global screening rate is halving-like because

$K$4

(Hwang et al., 20 Feb 2026).

In fixed-confidence BAI, LUCB- and Track-and-Stop-style methods remain central. Transfer LUCB (T-LUCB) applies LUCB logic to source–target transfer, choosing the target arm with the largest lower confidence bound and the strongest competitor by upper confidence bound, while sampling the source arms that contribute most to target uncertainty (Neopane et al., 2021). In spectral bandits, the asymptotically optimal strategy tracks the optimal proportions $K$5 and stops when a generalized likelihood-ratio statistic exceeds a threshold (Kocák et al., 2020). In constrained BAI with separate performance and feasibility tests, an LUCB-inspired algorithm alternates between performance sampling of the empirically best arm and its challenger, and feasibility sampling of the constraint most likely to violate the threshold, thereby eliminating arms by worse performance or infeasibility, whichever is easier (Cai et al., 12 Nov 2025).

Likelihood-ratio and top-two methods provide a second major family. TT-SPRT casts BAI as sequential composite hypothesis testing and compares the empirical best arm to its most plausible challenger using a generalized log-likelihood ratio $K$6, stopping when the top-vs-challenger statistic exceeds a threshold (Mukherjee et al., 2022). In the minimal-regret setting, Double KL-UCB combines an $K$7-UCB for regret-oriented leader selection and a $K$8-UCB for stopping and targeted exploration, with a stopping certificate based on the leader’s $K$9-lower confidence bound dominating the challenger’s $\mu_1,\dots,\mu_K$0-upper confidence bound (Yang et al., 2024). Posterior-driven approaches include Best Challenger with Thompson Exploration (BC-TE), which replaces forced exploration by disagreement between the empirical best arm and the posterior-best arm, and improved knowledge gradient (iKG), which changes the one-step look-ahead objective from “increase the estimated best mean” to “increase the probability of selecting the best arm,” leading to rate-optimality in the Gaussian fixed-budget setting studied there (Lee et al., 2023, Le et al., 2023).

4. Structure, context, transfer, and nonstationarity

Modern BAI increasingly exploits structure in the alternative set. In linear and generalized linear structured bandits, the mean reward takes the form

$\mu_1,\dots,\mu_K$1

and GSE estimates arm means from a joint regression model rather than treating arms as unrelated. In the linear case, with G-optimal design, the fixed-budget failure probability satisfies

$\mu_1,\dots,\mu_K$2

and the paper gives an analogous GLM bound with an extra $\mu_1,\dots,\mu_K$3 factor (Azizi et al., 2021). The same work makes the point that the objective is not to minimize regret, but to use a fixed number of samples to identify the optimal arm with highest possible probability.

In spectral bandits, the arm means satisfy a graph smoothness constraint

$\mu_1,\dots,\mu_K$4

This restricts the set of admissible alternatives, allows information to propagate through the graph structure, and can significantly reduce the characteristic time when the smoothness bound is informative (Kocák et al., 2020). A plausible implication is that “structure” in BAI is best understood as a restriction on confusing alternatives rather than merely as a regularizer on estimates.

Post-action context introduces a further distinction. In the non-separator case, the reward depends on both action and context, so Track-and-Stop can be extended by tracking the optimal arm proportions under the enriched arm-context model. In the separator case, the reward depends only on the context, and the correct object to track becomes the induced context-frequency vector in $\mu_1,\dots,\mu_K$5, leading to the G-tracking sampling rule (Shahverdikondori et al., 5 Feb 2025). This changes the geometry of optimal exploration: the learner effectively chooses a distribution over contexts indirectly by mixing arms.

Transfer BAI modifies observation rather than reward structure. Under additive transfer bandits, the learner samples only source arms, while each target mean is a known additive function of source means: $\mu_1,\dots,\mu_K$6 T-LUCB propagates source confidence intervals through the transfer map to obtain target confidence intervals, recovering ordinary LUCB as a special case when source and target coincide (Neopane et al., 2021).

Nonstationary structure appears in piecewise stationary linear bandits. There, the arm quality is the ensemble average over latent contexts,

$\mu_1,\dots,\mu_K$7

while both changepoints and context identities are hidden. PS$\mu_1,\dots,\mu_K$8BAI$\mu_1,\dots,\mu_K$9 runs a changepoint-aware subroutine and a naive subroutine in parallel; the former detects changepoints and aligns contexts, while the latter guarantees finite expected sample complexity. The paper proves that the expected sample complexity is optimal up to a logarithmic factor (Hou et al., 2024).

5. Robustness and privacy

Robust BAI addresses failures of classical assumptions. In fixed-budget stochastic bandits with adversarial corruptions, deterministic schedules can be attacked because the adversary knows exactly when each arm will be sampled. This is made explicit by attacks on Successive Halving, while randomized $$\mu_1>\mu_2\ge \cdots \ge \mu_K, \qquad \Delta_i := \mu_1-\mu_i.$$0 remains robust because the adversary knows only the sampling distribution, not the next arm itself (Zhong et al., 2020). The paper identifies a threshold phenomenon in the corruption per step $$\mu_1>\mu_2\ge \cdots \ge \mu_K, \qquad \Delta_i := \mu_1-\mu_i.$$1: below a threshold, best-arm identification succeeds with exponentially small failure probability; above a larger threshold, no algorithm can achieve a success probability tending to $$\mu_1>\mu_2\ge \cdots \ge \mu_K, \qquad \Delta_i := \mu_1-\mu_i.$$2 as $$\mu_1>\mu_2\ge \cdots \ge \mu_K, \qquad \Delta_i := \mu_1-\mu_i.$$3 (Zhong et al., 2020).

Differential privacy changes both algorithms and lower bounds. In fixed-confidence BAI under $$\mu_1>\mu_2\ge \cdots \ge \mu_K, \qquad \Delta_i := \mu_1-\mu_i.$$4-local and $$\mu_1>\mu_2\ge \cdots \ge \mu_K, \qquad \Delta_i := \mu_1-\mu_i.$$5-global DP, the lower bounds show two privacy regimes. In the low-privacy regime, the non-private KL-based characteristic time is recovered; in the high-privacy regime, the effective information is governed by Total Variation and the sample complexity scales like $$\mu_1>\mu_2\ge \cdots \ge \mu_K, \qquad \Delta_i := \mu_1-\mu_i.$$6 in the local model and $$\mu_1>\mu_2\ge \cdots \ge \mu_K, \qquad \Delta_i := \mu_1-\mu_i.$$7 in the global model (Azize et al., 2024). For Bernoulli global-DP BAI, the transportation cost becomes

$$\mu_1>\mu_2\ge \cdots \ge \mu_K, \qquad \Delta_i := \mu_1-\mu_i.$$8

and a private Top-Two algorithm matches the lower bound up to a multiplicative constant smaller than $$\mu_1>\mu_2\ge \cdots \ge \mu_K, \qquad \Delta_i := \mu_1-\mu_i.$$9 (Jourdan et al., 20 Oct 2025).

In fixed-budget linear bandits under $A_t$0-DP, DP-BAI uses a successive-elimination scheme with Laplace-perturbed empirical means and a maximum absolute determinant sampling principle. Its error probability decays exponentially in $A_t$1, with an exponent involving both the usual BAI hardness and a privacy penalty,

$A_t$2

where

$A_t$3

The corresponding upper and minimax lower bounds match order-wise in their dependence on the gaps, $A_t$4, and the complexity decomposition (Chen et al., 2024).

6. Bayesian, hybrid-data, and cross-objective viewpoints

Bayesian BAI changes the performance criterion from a worst-case guarantee over all instances to a prior-averaged guarantee. In fixed-confidence Bayesian BAI, the objective is

$A_t$5

Under Gaussian priors on the means, standard frequentist algorithms such as Track-and-Stop and Top-Two policies can be arbitrarily suboptimal: the paper proves that any frequentist $A_t$6-correct algorithm has

$A_t$7

The reason is that the prior gives nonzero mass to arbitrarily small gaps, making the frequentist $A_t$8 complexity non-integrable near $A_t$9 (Jang et al., 2024). A prior-aware elimination rule with early stopping matches a lower bound up to logarithmic factors.

A Bayesian perspective also changes fixed-budget analysis. Under a Gaussian random-effects prior $\tau$0, Random effect UCB Exploration (RUE) uses posterior means and variances,

$\tau$1

and the paper argues that classical exponentially decaying instance-dependent error rates can be artifacts of favorable realized gaps. Its Bayesian failure probability and simple regret bounds are of order $\tau$2, up to logarithmic factors (Zhu et al., 2024).

Hybrid-data settings produce a further departure from the classical model. With potentially biased offline data, the learner observes offline samples from $\tau$3 and online samples from $\tau$4, with possibly different means. The paper proves an impossibility result: without a valid per-arm bias bound $\tau$5, no $\tau$6-PAC algorithm can be universally adaptive in the sense of exploiting helpful offline data while remaining as robust as online-only methods when offline data are misleading (Yang et al., 29 May 2025). LUCB-H resolves this by combining online-only and bias-corrected offline confidence bounds and using the tighter one at each step.

Across these variants, several misconceptions are explicitly corrected by the literature. BAI is not equivalent to cumulative-regret minimization, and regret-minimizing algorithms such as LinUCB and GLM-UCB can perform poorly when repurposed for simple-regret objectives (Azizi et al., 2021). Deterministic elimination schedules are not intrinsically benign once corruptions are allowed (Zhong et al., 2020). Simultaneous observation of performance and constraints is restrictive in feasible BAI, because many applications require separate tests for efficacy and feasibility (Cai et al., 12 Nov 2025). A plausible implication is that the central unifying object in contemporary BAI is no longer only the arm mean vector, but the structured space of alternatives that remain statistically and operationally plausible under the observation model.