Gaussian Process Expected Improvement (GP-EI)

• GP-EI is a Bayesian optimization method that leverages Gaussian process surrogates to predict both the mean and uncertainty of expensive black-box functions.
• It selects new evaluation points by maximizing expected improvement, employing strategies like BPMI, BSPMI, and BOI to optimize the exploration–exploitation trade-off.
• Recent studies provide cumulative regret upper bounds and no-regret guarantees for GP-EI with standard kernels such as squared exponential and Matérn, reinforcing its practical and theoretical value.

Gaussian Process Expected Improvement (GP-EI) is a foundational acquisition function and sequential sampling principle in Bayesian optimization. It leverages a Gaussian process surrogate to predict both the mean and uncertainty of an expensive-to-evaluate black-box function and selects new evaluation points by maximizing the expected improvement over the best known value. GP-EI balances exploration (sampling where prediction uncertainty is high) and exploitation (refining regions that are already promising), and has become a widely adopted heuristic in machine learning, design of experiments, engineering, and applied sciences. The choice of incumbent (the “current best” reference value) in the EI formula is critical, especially in noisy observation regimes, and directly affects cumulative regret and convergence properties. Recent research has, for the first time, provided cumulative regret upper bounds for classic (non-modified) GP-EI, delineated the effects of multiple incumbent strategies, and connected these findings to rigorous no-regret guarantees for standard kernel families such as squared exponential and Matérn kernels (Wang et al., 21 Aug 2025).

1. Cumulative Regret Theory for GP-EI

The central theoretical advance in recent work is a rigorous derivation of cumulative regret upper bounds for the classic GP-EI algorithm in noisy settings under a Bayesian framework (where the unknown objective is a sample from a GP). The cumulative regret,

$R_T = \sum_{t=1}^T [f(x_t) - f(x^*)],$

measures the algorithm’s cumulative “suboptimality” over $T$ iterations. Using a refined analytical framework with super-martingale tools and information-theoretic measures (notably, the information gain $Y_T$), the regret bounds for GP-EI are derived for different choices of incumbent.

For the key variants:

• Best Posterior Mean Incumbent (BPMI):

$R_T = O\left(T^{1/2} \log^2(T)\right)$

for squared exponential kernels (with $a=2$), and more generally,

$$R_T = O\left(T^{(3\nu+2d)/(4\nu+d)} \log^{?}(T)\right)$$

for Matérn kernels of smoothness $\nu$ and dimension $d$.

• Best Sampled Posterior Mean Incumbent (BSPMI): Achieves similar sublinear growth (with a slightly larger constant or logarithmic factor due to restriction to the discrete set of previously evaluated points).
• Best Observed Incumbent (BOI): Satisfies sublinear cumulative regret in regimes where a “noisy” simple regret event holds frequently; if not, it still guarantees rapidly shrinking simple regret.

Sublinear $R_T$ implies that the per-step average regret $R_T/T$ vanishes as $T \to \infty$, characterizing no-regret performance (Wang et al., 21 Aug 2025).

2. Incumbent Selection: BPMI, BSPMI, and BOI

The choice of incumbent in the EI acquisition function is recognized as pivotal for both theoretical analysis and practical performance:

Incumbent	Definition / Update	Regret Bound	Computational Cost
BPMI	Global minimizer of GP mean	Sublinear (no-regret, optimal)	High (global optimization)
BSPMI	Best sampled mean so far	Sublinear (no-regret, near-opt.)	Low (discrete search)
BOI	Best noisy observed value	Sublinear (if few bad events)	Low, but can be brittle

• BPMI: Minimizes the GP posterior mean over the entire domain, ensuring rigorous control of the exploration/exploitation trade-off. Guarantees are strongest but entails full-domain optimization at each step.
• BSPMI: Restricts selection to the observed sampling set, which is computationally much cheaper but with only a minor loss in convergence rate.
• BOI: Ties the incumbent to the minimum of noisy sample values, making it potentially brittle—highly sensitive to particularly favorable or unfavorable noise realizations; its regret guarantees depend on the frequency of degraded “incumbent updates.”

Analytically, both BPMI and BSPMI provide robust, provably no-regret GP-EI algorithms under standard kernel assumptions (SE or Matérn), whereas BOI can be effective in practice at quickly reducing simple regret, but may not guarantee low cumulative regret in all noise regimes (Wang et al., 21 Aug 2025).

3. No-Regret Guarantees and Kernel Dependence

No-regret is achieved when $R_T/T \to 0$ as $T \to \infty$. For GP-EI with BPMI or BSPMI, this property is shown for common kernel classes:

• Squared Exponential Kernel: With $a=2$, the rate $R_T = O(T^{1/2} \log^2(T))$ suffices for $R_T/T \to 0$.
• Matérn Kernel: The bound depends polynomially on $T$ with exponent determined by $\nu$ and $d$; by selecting parameter $a$ appropriately, one maintains sublinear regret.
• BOI: The behavior is nuanced and hinges on the set of “bad incumbent updates”; if the number of such events is sublinear, regret remains sublinear; otherwise, simple regret still decreases quickly.

The analysis uses high-probability confidence intervals, connecting instantaneous regret to the posterior standard deviation at the newly sampled point, and sums these over time using information gain bounds.

4. Numerical Validation and Empirical Analysis

Benchmark experiments on synthetic functions (Branin 2D, Styblinski–Tang 2D, Camel 2D, Schwefel 2D, Rosenbrock 4D, Hartmann 6D) corroborate the theoretical predictions:

• For low noise levels, all variants achieve vanishing average regret.
• As noise increases, BPMI and BSPMI retain robust no-regret behavior, with empirical average $R_T / T$ decaying to zero, while BOI’s average regret may flatten or increase if deleterious incumbent updates occur early.
• Computational cost profiles highlight that BPMI’s full-domain minimization is expensive in high dimensions, motivating the use of BSPMI in large-scale or time-constrained applications.
• Simple regret (the best observed function value gap to the true optimum) generally reduces rapidly in all cases, but only BPMI/BSPMI guarantee that cumulative regret is minimized (Wang et al., 21 Aug 2025).

5. Practical Implications for Bayesian Optimization

Results from this line of work provide theoretical and operational guidance for real-world applications:

• Incumbent Choice: Practitioners should generally prefer BPMI or BSPMI as the incumbent in noisy settings to guarantee both robust exploration and provable no-regret. BSPMI is especially suitable when computational resources are limited.
• BOI Sensitivity: Use of BOI can be risky in high-noise environments due to potential early incumbent misidentification; in such cases, rapid initial simple regret decrease may not translate to good cumulative performance.
• Kernel Selection: The guarantees hold for standard kernels (SE, Matérn) under mild conditions, complementing information theoretic bounds on information gain from GP literature.
• Scalability: While BPMI supports the strongest theoretical rates, practical deployment for high-dimensional domains may prefer BSPMI. For problem classes where global optimization is feasible, BPMI is optimal.
• Algorithm Design: The results also clarify the intrinsic exploration–exploitation trade-off in GP-EI without requiring additional explicit exploration bonuses.

6. Context and Future Directions

Prior to these developments, the convergence properties and regret rates of standard GP-EI remained largely uncharacterized, particularly in the presence of noise. Recent advances (Tran-The et al., 2022, Wang et al., 25 Dec 2024, Wang et al., 16 Jan 2025, Wang et al., 21 Aug 2025) have now established the first non-asymptotic and asymptotic cumulative regret bounds for classical EI, elucidated the effects of incumbent strategy, and provided a rigorous connection between the sequential design principle and continuous bandit theory.

Extensions to frequentist settings, non-Gaussian noise, or alternative acquisition schemes (hierarchical, robust, modified for batch or composite objectives) remain active directions for research. The present work sets a benchmark for the theory and practice of GP-EI in Bayesian optimization, especially in noisy observation regimes.