Gaussian Process UCB

• Gaussian Process UCB is a method that models unknown functions with a Gaussian process and uses an upper confidence bound to balance exploration and exploitation.
• It leverages rigorous regret analysis and information gain measures to provide theoretical guarantees and near-optimal performance under varied noise and kernel settings.
• The approach has been extended to address multi-fidelity, non-stationary, safety-critical, and batch optimization scenarios, enhancing its practical versatility.

Gaussian Process Upper Confidence Bound (GP-UCB) is a foundational methodology in Bayesian optimization and kernelized bandit problems, providing a principled acquisition strategy that balances exploitation of known information with exploration of uncertain regions. GP-UCB and its extensions have attracted sustained interest due to their strong theoretical guarantees, practical efficacy across a range of optimization regimes, and versatility in structured, safe, and adaptive settings.

1. Algorithmic Principle and Posterior Construction

The GP-UCB algorithm sequentially optimizes an unknown function $f: X \rightarrow \mathbb{R}$ defined on a compact domain $X \subset \mathbb{R}^d$, under the assumption that $f$ is drawn from a zero-mean Gaussian process prior $\mathcal{GP}(0, k)$ with known positive-definite kernel $k$.

At each round $t$, after observing points $\{(x_i, y_i)\}_{i=1}^{t-1}$, where $y_i = f(x_i) + \varepsilon_i$ (with $\varepsilon_i$ typically sub-Gaussian or Gaussian noise), the GP posterior mean and variance at $x \in X$ are computed as

$X \subset \mathbb{R}^d$0

where $X \subset \mathbb{R}^d$1, $X \subset \mathbb{R}^d$2, and $X \subset \mathbb{R}^d$3 is a regularization/tuning parameter.

The UCB acquisition function is defined as

$X \subset \mathbb{R}^d$4

where $X \subset \mathbb{R}^d$5 is a confidence parameter, typically selected to ensure high-probability coverage for all $X \subset \mathbb{R}^d$6 at each $X \subset \mathbb{R}^d$7 (Wang et al., 2023, Amani et al., 2020, Iwazaki, 2 Jun 2025, Whitehouse et al., 2023).

2. Regret Analysis and Rates

The theoretical performance of GP-UCB is measured by cumulative regret

$X \subset \mathbb{R}^d$8

and, in some analyses, the simple regret $X \subset \mathbb{R}^d$9. Key regret bounds follow from high-probability uniform confidence intervals and the information gain $f$0, where

$f$1

For classical choices of $f$2, and under regularity assumptions on $f$3 and $f$4, the canonical result (Wang et al., 2023, Iwazaki, 2 Jun 2025, Whitehouse et al., 2023, Contal et al., 2015, Contal et al., 2016) is

$f$5

The scaling of $f$6 depends on the kernel and domain:

Kernel Type	$f$7 Scaling	$f$8 Order
Matérn$f$9	$\mathcal{GP}(0, k)$0	$\mathcal{GP}(0, k)$1 (Wang et al., 2023)
Squared Exponential	$\mathcal{GP}(0, k)$2	$\mathcal{GP}(0, k)$3 (Wang et al., 2023)

Recent advances have established that these bounds are minimax-optimal up to polylogarithmic factors and match known lower bounds for RKHS-constrained $\mathcal{GP}(0, k)$4 (Wang et al., 2023, Iwazaki, 2 Jun 2025, Whitehouse et al., 2023).

3. Algorithmic Variants and Extensions

Numerous extensions of GP-UCB have been developed to address specific modeling challenges:

• Noise-free setting: With noiseless observations, constant cumulative regret is achievable for SE and smooth Matérn kernels, i.e., $\mathcal{GP}(0, k)$5 for SE, and $\mathcal{GP}(0, k)$6 for Matérn when $\mathcal{GP}(0, k)$7 (Iwazaki, 26 Feb 2025).
• Randomized Exploration: Randomization of the confidence parameter, as in RGP-UCB (Gamma sampling) (Berk et al., 2020) and improved IRGP-UCB (shifted exponential sampling) (Takeno et al., 2024, Takeno et al., 2023), eliminates the need for an ever-increasing $\mathcal{GP}(0, k)$8, avoids late-stage over-exploration, and yields tighter Bayesian regret bounds $\mathcal{GP}(0, k)$9 on finite domains.
• Multi-fidelity: MF-GP-UCB exploits cheaper, biased surrogates of $k$0 to aggressively prune the search space, combining multi-level UCBs and fidelity selection to achieve regret scaling with the information gain on a much smaller subset (Kandasamy et al., 2016).
• Non-stationary Targets: WGP-UCB (Deng et al., 2021) and TV-GP-UCB/R-GP-UCB (Bogunovic et al., 2016) accommodate functions that evolve over time, via weighted/posterior discounting or Markovian temporal models, attaining sublinear regret up to variation dependent losses.
• Safety: SGP-UCB (Amani et al., 2020) augments GP-UCB with safety constraints encoded by an independent GP, ensuring all recommendations satisfy a high-probability safety threshold.
• Batch/Parallel: GP-UCB-PE (Contal et al., 2013) extends GP-UCB to the batch setting, offering simple regret reduction by a factor $k$1 with batch size $k$2.
• Local and Adaptive Search: MinUCB (Fan et al., 2024) and its lookahead variant LA-MinUCB replace global search with local UCB minimization, accelerating convergence in high dimensions while maintaining UCB-based optimality.
• Chaining and Cover-Based Methods: Chaining-UCB (Contal et al., 2015, Contal et al., 2016) leverages hierarchical cover trees to replace coarse discretizations, adaptively balancing exploration from covering number complexities.
• Adaptive Discretization and Computational Scaling: Ada-BKB (Rando et al., 2021) combines adaptive partition trees and Nyström GP surrogates to scale GP-UCB-style algorithms to large $k$3 and moderate dimensionality, without compromise to $k$4-type regret.

4. Regret Optimality, Information Gain, and the Resolution of Open Questions

The central technical contribution leading to regret-optimality of GP-UCB is the control of the posterior error—both bias term (deterministic approximation in the RKHS) and the stochastic estimation error—across the entire decision space and time horizon. Uniform error bounds that leverage empirical process theory, chaining, and regularization tuned to kernel eigendecay have enabled breakthrough results:

• Nearly optimal rates for polynomial eigendecay kernels: For Matérn kernels, careful tuning of regularization (ridge) and explicit use of separable Hilbert-space self-normalized concentration inequalities yield $k$5, which is sublinear for all $k$6, resolving the COLT open problem on optimality of GP-UCB (Whitehouse et al., 2023).
• Regret matches lower bounds: For both Matérn and SE kernels, recent analyses (Wang et al., 2023, Iwazaki, 2 Jun 2025, Whitehouse et al., 2023) show that GP-UCB achieves cumulative regret within logarithmic factors of the information-theoretic minimum.
• Bayesian and Frequentist Regimes: The regret bounds and optimality results are robust across Bayesian settings (GP prior truly generates $k$7) and frequentist RKHS-constrained settings with sub-Gaussian noise (Wang et al., 2023).
• Randomized UCB (IRGP-UCB) Tightens Bounds: By replacing $k$8 with a (possibly fixed) random draw, IRGP-UCB yields regret $k$9 on finite domains, while controlling over-exploration without intricate scheduling (Takeno et al., 2024, Takeno et al., 2023).

5. Practical Recommendations, Tuning, and Limitations

GP-UCB is easy to implement but requires choices for $t$0, kernel hyperparameters, discretization (in continuous domains), and, in some variants, parameters governing randomization or budget allocation. Key practical observations:

• Choice of $t$1: The theoretical prescription (e.g., $t$2) can be overly conservative in practice (Berk et al., 2020). Data-driven tuning or randomization is often preferable.
• Computational cost: Standard GP-UCB scales cubically with $t$3 for GP inference. Nyström/sketching (Rando et al., 2021), cover trees (Contal et al., 2015), and batch/parallelization strategies mitigate complexity.
• Exploration-exploitation tradeoff: Randomized UCB (RGP-UCB, IRGP-UCB) and hybrid random-exploration methods (Kim et al., 2024) allow finer-grained control of exploration without sacrificing regret guarantees.
• High-dimensional settings: All UCB-type algorithms degrade as dimensionality grows; the posterior uncertainty decays too slowly. Local BO, adaptive discretization, and chaining partially ameliorate this.
• Robustness to misspecification: In misspecified or agnostic kernel regimes, regret bounds degrade proportionally to approximation error $t$4, with algorithms like EC-GP-UCB and phased elimination providing minimax-optimal guarantees (Bogunovic et al., 2021).
• Safety-critical and non-stationary regimes: Modifications (e.g., SGP-UCB, WGP-UCB) allow operation under significant environmental volatility or safety constraints, with mild inflation to regret.

6. Summary of Key Theoretical and Empirical Outcomes

Recent work has solidified GP-UCB and its randomized variants as both theoretically regret-optimal and practically competitive:

• State-of-the-art regret: Sublinear regret with optimal rates for classical kernels; noise-free settings show constant regret for smooth cases.
• Comprehensive empirical validation: Across synthetic, benchmark, and real-world tasks (hyperparameter tuning, alloy design, non-convex optimization), GP-UCB and its randomization-friendly extensions outperform EI, PI, Thompson Sampling, and older UCB heuristics (Berk et al., 2020, Takeno et al., 2024, Takeno et al., 2023).
• Flexibility and extensibility: A broad ecosystem of variants enable application to batch, multi-fidelity, non-stationary, safety-constrained, and local search regimes; all are unified under the core UCB-acquisition principle.
• Resolution of optimality questions: Tight empirical process and chaining-based analyses have answered open questions regarding the asymptotic regret rate of vanilla GP-UCB and its randomized analogs (Wang et al., 2023, Whitehouse et al., 2023, Iwazaki, 2 Jun 2025, Iwazaki, 26 Feb 2025).

7. Table: Summary of GP-UCB Regret Bounds by Regime

Setting	Regret Bound	Notes / Kernel	Reference
Noisy, SE kernel	$t$5	Optimal (up to logs)	(Wang et al., 2023)
Noisy, Matérn$t$6 kernel	$t$7	Matches minimax up to logs	(Wang et al., 2023)
Noise-free, SE kernel	$t$8	Constant regret	(Iwazaki, 26 Feb 2025)
Noise-free, Matérn$t$9, $\{(x_i, y_i)\}_{i=1}^{t-1}$0	$\{(x_i, y_i)\}_{i=1}^{t-1}$1		(Iwazaki, 26 Feb 2025)
Noise-free, Matérn$\{(x_i, y_i)\}_{i=1}^{t-1}$2, $\{(x_i, y_i)\}_{i=1}^{t-1}$3	$\{(x_i, y_i)\}_{i=1}^{t-1}$4	Proven optimal up to constants	(Iwazaki, 26 Feb 2025)
Randomized IRGP-UCB, finite $\{(x_i, y_i)\}_{i=1}^{t-1}$5	$\{(x_i, y_i)\}_{i=1}^{t-1}$6	Removes $\{(x_i, y_i)\}_{i=1}^{t-1}$7 penalty	(Takeno et al., 2024)

GP-UCB and its extensions remain the leading paradigm for kernel bandit optimization, offering rigorous regret guarantees, problem-adaptive flexibility, and strong empirical performance across the spectrum of Bayesian optimization tasks.