Meta-UCB: Meta-Level UCB Algorithms

• Meta-UCB is a family of algorithms that extend traditional UCB principles by integrating meta-level strategies for sequential decision-making.
• It combines techniques from robust meta-Bayesian optimization, algorithm selection, and pure exploration to improve performance in complex settings.
• Meta-UCB methods enable adaptive tuning of exploration-exploitation balances using data-driven meta-priors and performance-based algorithm aggregation.

Meta-UCB encompasses a family of algorithms and abstractions that generalize or extend the classical Upper Confidence Bound (UCB) principle for sequential decision-making and optimization. These methodologies arise in settings ranging from robust meta-Bayesian optimization, combiners of multiple stochastic bandit algorithms, finite-budget pure exploration, to meta-learned index policies for bandits. The unifying theme is the construction of meta-level UCB indices or frameworks for either leveraging prior tasks, orchestrating algorithmic experts, or learning problem-tailored UCB-type exploration-exploitation strategies.

1. General Frameworks and Definitions

Meta-UCB is defined in multiple, problem-specific ways, unified by the idea of operating a UCB principle at a meta-level.

• Meta-Bayesian Optimization: In robust meta-Gaussian Process UCB (RM-GP-UCB), Meta-UCB refers to using a weighted combination of previous task posteriors to initialize a Gaussian process (GP) prior, enabling robust transfer learning for the black-box function optimization setting (Dai et al., 2022).
• Combining Algorithms as Arms: In the context of bandit algorithm selection, Meta-UCB treats each bandit algorithm (or expert) as an arm in a higher-level multi-armed bandit, applying a UCB-style scheme to select which algorithm to invoke (Cutkosky et al., 2020).
• Pure Exploration: In large-scale best-arm identification, Meta-UCB denotes a class of UCB-type algorithms where each arm’s index is its empirical mean plus an exploration bonus dependent only on its own sample size; the algorithm samples up to a fixed budget and then returns the arm with the highest number of samples as the best (Li et al., 27 Nov 2025).
• Meta-learning UCB Indices: A meta-learning approach tunes the UCB index function itself (either in numeric parameter space or in symbolic/formulaic space) using prior knowledge of the bandit problem distribution, resulting in data-driven, problem-specific UCB variants (Maes et al., 2012).

2. Meta-UCB in Robust Meta-Bayesian Optimization

The RM-GP-UCB algorithm is a principled approach for meta-Bayesian optimization leveraging data from $M$ previous tasks with potentially harmful dissimilarity to the target. It operates as follows (Dai et al., 2022):

• Meta-prior Formation: For each previous task $f^i$, form a GP posterior $\overline{\mu}_i(x), \overline{k}_i(x, x')$ using observed data. Aggregate these via nonnegative weights $w_i$ into a meta-prior mean and covariance:

$$\mu_0(x) = \sum_{i=1}^M w_i \overline{\mu}_i(x), \quad k_0(x, x') = \sum_{i=1}^M w_i^2 \overline{k}_i(x, x')$$

• Optimization: Start BO on the target function $f$ with the meta-GP prior, updating after each target-task query.
• Acquisition Rule: At iteration $t$, select $$x_t = \arg\max_x \mu_{t-1}(x) + \sqrt{\beta_t} \sigma_{t-1}(x)$$ with $\beta_t$ determined by quantifying uncertainty/information gain.
• Theoretical Robustness: Cumulative regret $R_T$ is bounded by terms depending on both task similarity and GP information gain, guaranteeing asymptotic no-regret even with many dissimilar prior tasks. By contrast, RM-GP-Thompson Sampling (RM-GP-TS) is less robust.
• Adaptive Weighting: The weights $f^i$0 are adapted online via mirror descent (entropically regularized Follow-the-Leader) procedures using empirical upper-bounds on task dissimilarity, so harmful tasks are downweighted.

3. Meta-UCB for Combining Candidate Algorithms

Meta-UCB algorithms are used to select among multiple adaptive learning algorithms or bandit subroutines, treating each as an arm in a meta-level bandit problem (Cutkosky et al., 2020, Latypov et al., 26 Oct 2025). Principal elements:

• Algorithm: At each round, the Meta-UCB index for each base algorithm $f^i$1 is computed from its average observed reward and its own regret or variance bound. The algorithm selected is that with the highest index after a suitable penalty (pruning arms whose empirical pseudo-regret exceeds its hypothesized bound).
• Regret Bounds: The meta-regret is at most three times the regret parameter picked for the best base learner in hindsight, matching or improving bounds from adversarial or corral-type algorithms.
• Advantages Over CORRAL: Meta-UCB imposes no stability condition and allows for non-uniform exploration rates and priors (e.g., via $f^i$2 per arm), yielding strictly finer trade-offs and recovery of optimal regret under misspecification.
• Extensions: With budget constraints (only $f^i$3 out of $f^i$4 experts can be trained per round), the extension M-LCB achieves regret $f^i$5 when the internal regret of each expert is $f^i$6 (Latypov et al., 26 Oct 2025).

4. Meta-UCB for Large-Scale Pure Exploration

The Meta-UCB approach in best-arm identification under large $f^i$7 and minimal distributional assumptions is as follows (Li et al., 27 Nov 2025):

• Algorithmic Principle: Each arm $f^i$8 is assigned an index $f^i$9, where $\overline{\mu}_i(x), \overline{k}_i(x, x')$0 is a non-increasing exploration bonus specific to $\overline{\mu}_i(x), \overline{k}_i(x, x')$1, the sample count for arm $\overline{\mu}_i(x), \overline{k}_i(x, x')$2.
• Selection and Stopping: At each step, sample the arm with the highest index. Upon exhausting