Dynamic Arm Selection Techniques

• Dynamic arm selection is an adaptive decision-making process that uses multi-armed bandit frameworks to sequentially select actions for optimal information gain.
• It employs methodologies like Successive Elimination, KL-UCB, and group testing to minimize sample complexity and regret under both stochastic and non-stochastic settings.
• Applications span diverse domains such as quantum simulation, clinical trials, and robotic control, highlighting its practical impact on efficient resource allocation.

Dynamic arm selection refers to adaptive algorithms and principled strategies for sequentially choosing among a set of candidate arms (actions, treatments, recommendations, or generator operators), with the goal of efficient information acquisition, optimal decision-making, or resource allocation under uncertainty, constraints, or structural side information. The term spans stochastic and non-stochastic multi-armed bandit (MAB) frameworks, combinatorial bandits, contextual bandits, and numerous domain-specific applications, including quantum simulation, clinical trials, algorithmic fairness, preference shaping, and actuation control. Research on dynamic arm selection targets minimization of sample complexity, regret, or error rates by focusing exploration and exploitation where it is most informative or impactful.

1. Formal Problem Settings and Objectives

Dynamic arm selection arises in several canonical problem classes:

• Best-Arm Identification (BAI) and Threshold-Based Bandits: The learner aims to identify the arm with the maximal mean reward, or one satisfying a threshold relation (e.g., minimal index with $\mu_k \ge \tau$). Classical regret minimization and sample-efficient $\delta$-correct identification of the optimal arm under a fixed confidence or fixed budget constraint are both studied (Varude et al., 2 Sep 2025, Agrawal et al., 2019).
• Contextual and Group-Fair Bandits: Arms are associated with context features and possibly grouped by sensitive attributes. The objective is to maximize cumulative reward while enforcing group fairness and correcting for societal or measurement bias (Schumann et al., 2019).
• Combinatorial and Structured Bandits: At each round, the action is a subset (super-arm) of base arms, often under cardinality or structural constraints. Efficient super-arm selection must avoid exponential computational cost and scale to large base-arm universes (Mukherjee et al., 14 Oct 2024).
• Non-stationary and Preference-Shaping Bandits: The reward-generating environment or the population's composition evolves over time, often in response to the learner's actions. The objective shifts from maximizing static reward to dynamically shaping future preference distributions (Nadkarni et al., 29 Feb 2024).
• Resource-Constrained and Acquisition-Cost Models: Querying new arms may involve non-trivial acquisition costs, dynamically changing the set of available actions and the exploration–exploitation calculus (Kalvit et al., 2021).
• Physical Control Systems: In robotics and cyber-physical systems, arm selection may refer to real-valued actuator choices (e.g., arm length in UAVs) that modulate geometric and dynamic properties of the platform (Kumar et al., 2020).

2. Algorithmic Foundations and Methodologies

The technical literature introduces several adaptive, sample-efficient arm selection mechanisms. A selection of canonical methodologies includes:

• Successive Elimination and Halving: Iterative schemes prune arms whose empirical performance is provably suboptimal, reallocating measurement or computational budget to surviving candidates. For stochastic BAI, Successive Elimination allocates measurement effort in rounds and discards arms whose upper confidence bounds are dominated, achieving sample complexity $O(\sum_{i\ne M} \Delta_i^{-2} \ln(1/\delta))$ (Huang et al., 18 Sep 2025). In non-stochastic settings, Successive Halving adaptively allocates resources in a fixed budget setting and eliminates half the arms per round, providing robust guarantees even without i.i.d. assumptions (Jamieson et al., 2015).
• KL-UCB and Index Policies: Dynamic upper/lower confidence bounds, often based on Kullback-Leibler divergence, are used to adaptively focus exploration on arms near critical decision boundaries (e.g., closest to a threshold in monotonic bandits) (Varude et al., 2 Sep 2025). Algorithms adjust sampling toward the arms whose inclusion/exclusion decision remains uncertain, often requiring only order-$O(1)$ per-round computation.
• Group Testing and Quantized Thompson Sampling: In high-dimensional combinatorial settings, adaptive group testing procedures and quantized posterior sampling enable near-optimal super-arm selection with only logarithmic-in-$m$ computational cost, matching the regret of exponential-oracle algorithms while remaining tractable (Mukherjee et al., 14 Oct 2024).
• Block-Randomized Bayesian Arm Selection: In adaptive clinical trials, block randomization in conjunction with Bayesian posterior update and probabilistic dropout rules allows ongoing allocation adaptation and early stopping of arms, embedding resource reallocation in the trial design (Arjas et al., 2021).
• Max-Utility and Submodular Maximization: For countably infinite or structured arm sets, candidate-arm selection is formulated as a submodular maximization problem (e.g., maximizing the utility of the candidate set under preference probabilities), using distributed or greedy approximation algorithms to reduce the effective arm set before downstream bandit algorithms are run (Parambath et al., 2021).
• Preference Shaping in Non-Stationary Bandits: In dynamic populations, "explore-then-commit" or Thompson Sampling policies are derived with the minimization of population-regret, aiming to direct the aggregate preference (e.g., Polya urn or voter model dynamics) toward a target arm (Nadkarni et al., 29 Feb 2024).
• Physical Dynamic Actuation: In platforms like morphing UAVs, arm selection corresponds to real-time dynamic control of structural parameters (e.g., arm length), integrated via feedback laws enabling additional degrees of actuation for maneuverability and stabilization (Kumar et al., 2020).

3. Sample Complexity and Regret Analysis

A core focus is the rigorous quantification of statistical or computational efficiency in dynamic arm selection:

• In monotonic threshold bandits, regret is shown to grow as $O(\ln T)$, with constants determined only by the (Bernoulli KL) divergence between critical adjacent arms and the threshold (Varude et al., 2 Sep 2025).
• Best-arm algorithms under heavy-tailed reward distributions achieve sample complexity proportional to the inverse squared reward gap, plus potentially loose moments, with batch processing delivering optimal-to-nearly-optimal computational trade-offs (Agrawal et al., 2019).
• For dynamic acquisition cost models, necessary and sufficient rates for sublinear regret are quantified in terms of the cumulative probability of acquiring optimal-type arms, with $O(\log n)$ regret achievable only if $\sum_t \alpha(t) = \infty$ (Kalvit et al., 2021).
• Fair group-sensitive bandits incur an adjusted regret of $O(T^{2/3})$ (for two groups), reflecting the statistical cost of simultaneously de-biasing group feedback and learning arm parameters (Schumann et al., 2019).

Problem Variant	Sample/Regret Scaling	Dominant Arms
Monotonic Threshold Bandit	$O(\ln T)$	Arms adjacent to $\tau$
Heavy-Tailed Best-Arm (BAI)	$O(\sum_{i\ne M} \Delta_i^{-2} \ln(1/\delta))$	All non-best arms (gap $\Delta_i$)
Group-Fair Contextual Bandit	$O(T^{2/3})$	All groups, per-group bias
Combinatorial (Group Test+QTS)	$O(m/\Delta_{\min} \log K \log T)$	Base arms with minimal gap
Dynamic Acquisition Cost Bandit	$O(\log n / \alpha(n))$	Driven by reservoir decay rate

4. Dynamic Arm Selection Mechanisms Across Domains

Dynamic arm selection principles recur in diverse applied settings:

• Quantum Simulation: In Adaptive Variational Quantum Algorithms (AVQAs), generator selection is mapped to BAI, with Successive Elimination orders of magnitude more efficient than uniform-precision baselines—achieving up to 93% reduction in measurements on chemistry benchmarks while preserving energy accuracy (Huang et al., 18 Sep 2025).
• Clinical Trials: Block-randomized Bayesian adaptive allocation enables early termination of inferior treatment arms, continuous updating of posterior success probabilities, and adaptive resource refocusing subject to statistical error rate controls (Arjas et al., 2021).
• Recommendation and Query Systems: Contextual submodular candidate selection pipelines filter potentially infinite arm sets to a high-utility shortlist (e.g., by semantic similarity), upon which regret-minimizing bandit algorithms operate (Parambath et al., 2021).
• Robotic Control: Real-time actuation of variable-geometry parameters, such as arm length in quadcopters, augments conventional control inputs, improving disturbance rejection and maneuverability by embedding selection in a feedback loop (Kumar et al., 2020).
• Preference Shaping: In evolving and reactive populations, bandit-based adaptive arm selection directly steers the distributional state toward desired configurations, leveraging dynamic opinion-reinforcement models (Nadkarni et al., 29 Feb 2024).

5. Theoretical Guarantees and Empirical Performance

Across the literature, optimality proofs and empirical validation are provided:

• Change-of-measure and information-theoretic lower bounds for monotonic bandits demonstrate that, aside from initialization, nearly all exploration can be adaptively focused on at most two arms near the threshold, yielding tight matching upper and lower regret bounds (Varude et al., 2 Sep 2025).
• In BAI with heavy tails, explicit lower/upper sample complexity characterizations match up to $o(\ln(1/\delta))$ corrections, provided minimal moment constraints hold (Agrawal et al., 2019).
• Empirical evaluations confirm that adaptive algorithms (SE in AVQAs, Successive Halving for non-stochastic best-arm, block-Bayesian clinical allocation, group-testing combinatorial bandits) achieve orders-of-magnitude gains over uniform or naïve selection, and consistently match theoretical prescriptions across synthetic and real datasets (Huang et al., 18 Sep 2025, Jamieson et al., 2015, Arjas et al., 2021, Mukherjee et al., 14 Oct 2024, Schumann et al., 2019).

6. Extensions, Limitations, and Open Directions

Multiple research frontiers remain:

• Identification of optimal arm relative to unknown monotonic order, multi-threshold, or multi-quantile bandits (Varude et al., 2 Sep 2025).
• Highly structured or nonparametric reward families with shape or sparsity constraints.
• Endogenous arm reservoirs, adversarial arm generation, or costly acquisition—rate-limited by underlying arm diversity (Kalvit et al., 2021).
• Generalization to non-stationary, networked, or dynamic population models for preference shaping (Nadkarni et al., 29 Feb 2024).
• Integration of dynamic selection in high-dimensional combinatorial or control-input spaces without exponential computation (Mukherjee et al., 14 Oct 2024, Kumar et al., 2020).
• Fairness and bias mitigation regimes with more than two protected groups, complex feedback or delay structures (Schumann et al., 2019).

A plausible implication is that as application domains increasingly feature structural knowledge (monotonicity, grouping, combinatorial feasibility), domain-specific dynamic arm selection mechanisms will deliver dramatic gains in efficiency and fairness, provided careful algorithmic design and theoretical analysis anchor their deployment.