---
title: Multi-Fidelity UCB Optimization
url: https://www.emergentmind.com/topics/multi-fidelity-upper-confidence-bound-ucb
type: topic
---

# Multi-Fidelity UCB Optimization

Searching arXiv for the cited MF-UCB and related multi-fidelity UCB papers to ground the article.
Multi-Fidelity Upper Confidence Bound (MF-UCB) denotes a class of optimistic, cost-aware sequential decision procedures for optimization and bandit problems in which the target quantity is available at a highest fidelity, while cheaper lower fidelities provide biased or approximate information. Across its main formulations, MF-UCB combines confidence bounds with explicit fidelity selection rules so that low-cost evaluations screen large portions of the search space or arm set, and expensive high-fidelity evaluations are concentrated on candidates that remain plausible after lower-fidelity filtering. In the multi-armed bandit setting, this is formalized through bias-bounded approximations with fidelity-dependent costs [1610.09726]. In Gaussian-process bandit optimization, the same principle appears as fidelity-wise upper bounds on the true objective, aggregated into a single optimistic index [1603.06288]. More recent work has also implemented MF-UCB with multi-fidelity Gaussian processes of Kennedy–O’Hagan type, replacing separate single-fidelity posteriors with a co-kriging surrogate and adding tunable cost-ratio controls [2508.01013].

## 1. Core concept and problem setting

The defining feature of MF-UCB is the joint treatment of three quantities: uncertainty, approximation error, and evaluation cost. The highest fidelity is the target of interest and is the most expensive to query. Lower fidelities are cheaper, but they are not exact substitutes; they are biased approximations or simplified models. MF-UCB uses upper confidence bounds to propagate lower-fidelity information into decisions about where or which arm to evaluate next, while explicitly accounting for fidelity-specific error and cost [1610.09726].

In the stochastic multi-armed bandit formulation, there are $K$ arms and $M$ fidelities. Evaluating arm $k$ at fidelity $m$ incurs cost $\lambda^{(m)}$, with lower fidelities satisfying $\lambda^{(m)} < \lambda^{(M)}$. The highest fidelity has mean $\mu_k$, while lower fidelities have means $\mu_k^{(m)}$ that obey fidelity-wise bias bounds
$$
\bigl| \mu_k^{(m)} - \mu_k \bigr| \le \zeta^{(m)},
$$
with nondecreasing costs and nonincreasing bias bounds across fidelities [1610.09726].

In Gaussian-process optimization, the objective is to maximize an expensive black-box function $f \equiv f^{(M)}$ on a domain $\mathcal{X}$, given access to lower-fidelity approximations $f^{(m)}$. Here the discrepancy is expressed through uniform approximation bounds,
$$
\|f^{(M)} - f^{(m)}\|_\infty \le \zeta^{(m)},
$$
and each fidelity is assigned a GP posterior used to form a valid upper bound on the true objective [1603.06288].

A plausible implication is that MF-UCB is best understood not as a single algorithm but as a design pattern: optimism is retained, but the optimistic index is modified so that cheap, imperfect information can be exploited without treating it as exact.

## 2. Multi-armed bandit MF-UCB

In the multi-fidelity multi-armed bandit setting, MF-UCB extends classical UCB by constructing fidelity-specific upper bounds on the true arm mean. With empirical mean $\hat{\mu}_{k,m}(t)$ and confidence radius $\beta_{k,m}(t)$, a canonical fidelity-wise index is
$$
U_{k,m}(t) = \hat{\mu}_{k,m}(t) + \beta_{k,m}(t) + \zeta^{(m)}.
$$
The additive $\zeta^{(m)}$ term compensates for the worst-case bias of fidelity $m$, so each $U_{k,m}(t)$ serves as an upper bound on the highest-fidelity mean $\mu_k$ [1610.09726].

Because every fidelity yields a valid upper bound, the arm-level optimistic index is aggregated by taking the tightest one:
$$
U_k(t) = \min_{m \in \{1,\dots,M\}} U_{k,m}(t).
$$
The arm selected at round $t$ is the one with the largest aggregated upper bound,
$$
k_t \in \arg\max_k U_k(t).
$$
This “minimum across fidelities” is the central structural move in MF-UCB: low fidelities are used whenever they suffice to produce a tight certificate, and higher fidelities are invoked only when lower-fidelity uncertainty or bias becomes the limiting factor [1610.09726].

Fidelity selection is governed by a promotion rule. The guiding intuition is that if the statistical uncertainty at fidelity $m$ is already below the fidelity’s inherent bias, then additional samples at that fidelity cannot materially sharpen the bound. A concrete rule is to choose, for the selected arm, the smallest fidelity satisfying
$$
\beta_{k,m}(t) \ge \zeta^{(m)},
$$
and otherwise promote to the highest fidelity. The synthesized description of the method states that MF-UCB uses lower fidelities to quickly eliminate suboptimal arms and reserves expensive experiments for a small set of promising candidates; it also states that the method is nearly optimal under certain conditions [1610.09726].

The associated objective is budget-aware regret minimization under a total cost budget $\Lambda$. Costs affect performance through the total number of possible plays and through the algorithm’s ability to substitute cheap lower-fidelity observations for expensive top-fidelity ones. This suggests that the principal advantage of bandit MF-UCB emerges when the lower fidelities are both substantially cheaper and sufficiently informative.

## 3. Gaussian-process MF-GP-UCB

MF-GP-UCB, introduced for Gaussian-process bandit optimization, translates the same principle into continuous black-box optimization. The setting assumes $M$ fidelities, fidelity-dependent query costs $\lambda^{(m)}$, and approximation bounds $\zeta^{(m)}$ satisfying $\zeta^{(1)} > \zeta^{(2)} > \cdots > \zeta^{(M)} = 0$ [1603.06288].

At each fidelity $m$, a separate GP posterior is maintained using only the data collected at that fidelity. If $\mu_t^{(m)}(x)$ and $\sigma_t^{(m)}(x)$ denote the posterior mean and standard deviation, the fidelity-wise upper confidence bound on the true objective is
$$
UCB_t^{(m)}(x) = \mu_{t-1}^{(m)}(x) + \sqrt{\beta_t}\,\sigma_{t-1}^{(m)}(x) + \zeta^{(m)}.
$$
The approximation term $\zeta^{(m)}$ converts a confidence bound on $f^{(m)}$ into a confidence bound on $f^{(M)} = f$. The combined acquisition is then the pointwise minimum across fidelities,
$$
\varphi_t(x) = \min_{m \in \{1,\dots,M\}} UCB_t^{(m)}(x),
$$
and the next query location is chosen by
$$
x_t = \arg\max_{x \in \mathcal{X}} \varphi_t(x).
$$
This retains the optimistic logic of GP-UCB while incorporating multiple biased approximations [1603.06288].

Fidelity selection is handled by uncertainty thresholds $\gamma^{(m)}$. After choosing $x_t$, the method queries the smallest fidelity for which the scaled uncertainty remains sufficiently large:
$$
m_t = \min\{ m \in \{1,\dots,M\} : \sqrt{\beta_t}\sigma_{t-1}^{(m)}(x_t) \ge \gamma^{(m)} \text{ or } m=M \}.
$$
The interpretation given in the source is that the algorithm should not over-invest at a low fidelity once its uncertainty is already smaller than the bias $\zeta^{(m)}$; at that point escalation becomes appropriate [1603.06288].

The theoretical analysis emphasizes “good sets” in which the top-fidelity queries concentrate. For $M=2$, the good set is defined as
$$
\mathcal{X}_{\text{good}} = \{x \in \mathcal{X}: f(x^*) - f^{(1)}(x) \le \zeta^{(1)} \},
$$
with an inflated version $\mathcal{X}_{\text{good}}^\gamma$ incorporating the threshold $\gamma^{(1)}$. The analysis states that almost all top-fidelity queries are confined to such sets, so the simple-regret bound depends on the maximum information gain over a reduced region rather than the entire domain. The resulting improvement is described as replacing $\gamma_n(\mathcal{X})$ with $\gamma_n(H^{(M)})$ or a mild dilation thereof, while preserving the same $1/\sqrt{n}$ rate [1603.06288].

A common misconception is that MF-GP-UCB is simply co-kriging with UCB. The formulation summarized here explicitly states the opposite: it maintains a separate GP posterior for each fidelity and introduces cross-fidelity coupling only through the discrepancy bounds $\zeta^{(m)}$ rather than a joint multi-output GP [1603.06288].

## 4. MF-UCB with multi-fidelity Gaussian processes

A later implementation modifies the MF-UCB idea by replacing separate per-fidelity GPs with a two-fidelity Kennedy–O’Hagan co-kriging model. In this formulation, the low fidelity is modeled as
$$
Z_{\text{low}}(x) \sim GP(m_{\text{low}}(x), \kappa_{\text{low}}(x,x')),
$$
with $m_{\text{low}}(x)=0$ and squared exponential $\kappa_{\text{low}}$, while the discrepancy process is
$$
\delta(x) \sim GP(m_\delta(x), \kappa_\delta(x,x')),
$$
with $m_\delta(x)=0$ and squared exponential $\kappa_\delta$. The high fidelity is defined autoregressively by
$$
Z_{\text{high}}(x) = \rho \, Z_{\text{low}}(x) + \delta(x),
$$
where $\rho$ is a scalar scaling hyperparameter learned from data [2508.01013].

Under mixed-fidelity observations, the posterior mean and variance at high fidelity are
$$
\mu_{\text{high}}(x) = \rho \mu_{\text{low}}(x) + \kappa_\delta(x, X_{\text{high}}) K_\delta^{-1}\bigl[y_{\text{high}} - \rho \mu_{\text{low}}(X_{\text{high}})\bigr],
$$
and
$$
\sigma_{\text{high}}^2(x) = \rho^2 \sigma_{\text{low}}^2(x) + \kappa_\delta(x,x) - \kappa_\delta(x, X_{\text{high}}) K_\delta^{-1}\kappa_\delta(X_{\text{high}},x).
$$
Nested designs $X_{\text{high}} \subseteq X_{\text{low}}$ are preferred; for non-nested designs, $\mu_{\text{low}}(X_{\text{high}})$ is used to form the discrepancy target. The formulas are presented as essentially noise-free, with diagonal jitter added to the kernel matrices if noise handling is required [2508.01013].

Within this MF-GP surrogate, the implemented MF-UCB acquisition is
$$
\alpha_{\text{low}}(x) = \mu_{\text{low}}(x) + \sqrt{\beta}\,\sigma_{\text{low}}(x) + \zeta(x),
$$
$$
\alpha_{\text{high}}(x) = \mu_{\text{high}}(x) + \sqrt{\beta}\,\sigma_{\text{high}}(x),
$$
with the local inter-fidelity error estimated by
$$
\zeta(x) = \|\mu_{\text{high}}(x) - \mu_{\text{low}}(x)\|.
$$
Candidate selection uses the combined bound
$$
x_t = \arg\max_x \min(\alpha_{\text{low}}(x), \alpha_{\text{high}}(x)),
$$
and fidelity selection is determined through
$$
\gamma(x_t) = \zeta(x_t)\sqrt{\lambda_2/\lambda_1},
$$
querying the low fidelity if $\sqrt{\beta}\sigma_{\text{low}}(x_t) > \gamma(x_t)$ and otherwise the high fidelity [2508.01013].

The implemented procedure, called MF-GPR-UCB, retrains the MF-GP at each iteration by optimizing kernel hyperparameters via marginal likelihood, uses L-BFGS multi-start to optimize the acquisition, and recommends reserving a small number of high-fidelity evaluations for final verification at the best $\mu_{\text{high}}$ location. The same source explicitly states that it does not claim formal regret guarantees for MF-GPR-UCB with MF-GPs [2508.01013].

This development marks an important methodological distinction. Earlier MF-GP-UCB uses independent per-fidelity GPs and analytic bias bounds $\zeta^{(m)}$ [1603.06288], whereas the newer MF-GPR-UCB uses an explicit inter-fidelity correlation model and estimates $\zeta(x)$ pointwise from the discrepancy between MF-GP means [2508.01013].

## 5. Tunability, cost ratios, and alternative acquisition design

A prominent practical theme in recent multi-fidelity UCB work is tunability through cost ratios. In the MF-GPR-UCB formulation, the tunable parameters are $\beta$, $\lambda_1$, and $\lambda_2$. Fixed $\beta$ choices are divided into exploitative settings $\beta \in \{0.5,1\}$ and explorative settings $\beta \in \{3,5\}$, while an adaptive schedule borrowed from earlier work is
$$
\beta_t = \sqrt{0.2 d \log(2t)}.
$$
The cost ratio $\Lambda = \lambda_1/\lambda_2$ controls high-fidelity usage through the threshold $\gamma(x_t)=\zeta(x_t)\sqrt{\lambda_2/\lambda_1}$ [2508.01013].

The same study introduces a proximity-based acquisition strategy as an alternative to per-fidelity acquisition functions. It selects the next point using only the high-fidelity acquisition,
$$
x_t = \arg\max_x \alpha_{\text{high}}(x),
$$
and then chooses the fidelity from the minimum distance to the existing low-fidelity design,
$$
d_{\min}(x_t;X_{\text{low}})=\min_{x_i \in X_{\text{low}}}\|x_t-x_i\|,
$$
with proximity radius
$$
r = \lambda_1/\lambda_2.
$$
If $d_{\min}(x_t;X_{\text{low}}) > r$, the algorithm queries the low fidelity; otherwise it queries the high fidelity. The stated goal is to use a single acquisition function and select the fidelity by a proximity criterion that depends on local low-fidelity sample density and a tunable cost ratio [2508.01013].

The paper also evaluates a fidelity-weighted baseline. For counts $n_1$ and $n_2$ of low- and high-fidelity evaluations, the cost penalties are
$$
C_{\text{low}} = (\lambda_1/\lambda_2)(n_1+1)+n_2,\qquad
C_{\text{high}} = (\lambda_1/\lambda_2)n_1+(n_2+1),
$$
and the modified acquisitions are
$$
\alpha_{\text{low}}^{fw}(x)=\alpha_{\text{low}}(x)-[1/(n_{\text{iter}}C_{\text{low}})],
$$
$$
\alpha_{\text{high}}^{fw}(x)=\alpha_{\text{high}}(x)-[1/(n_{\text{iter}}C_{\text{high}})].
$$
The candidate is selected by maximizing the larger of the two penalized acquisitions, and the fidelity is then chosen by whichever penalized acquisition is larger at that candidate [2508.01013].

The reported comparison gives three distinct behaviors. The proximity-based strategy is described as providing smooth, predictable high-fidelity-usage control across $\Lambda \in [0,1]$ with tight distributions. MF-GPR-UCB shows a smoother average trend than the fidelity-weighted approach, but also wide variance and sometimes elevated high-fidelity usage even at low $\Lambda$. The fidelity-weighted approach is described as sharply sensitive to the cost ratio, with abrupt jumps in high-fidelity usage and poor low-to-high information exchange [2508.01013].

A plausible implication is that, in practice, MF-UCB variants differ at least as much in their fidelity-allocation mechanism as in their optimistic index.

## 6. Empirical behavior, applications, and limitations

The recent benchmark study compares MF-GPR-UCB, the proximity-based strategy, and the fidelity-weighted approach on Forrester, Bohachevsky, toy enzyme, Himmelblau, Oregonator Hopf search, and dynamic ammonia catalysis. Its overall summary is that proximity-based acquisition consistently provided predictable control of high-fidelity usage with competitive regret, often the lowest high-fidelity fraction used; MF-GPR-UCB showed good balance in some cases but inconsistency in others; and the fidelity-weighted approach tended to rely heavily on high fidelity and was very sensitive to the cost ratio [2508.01013].

Several task-specific outcomes are reported. On Forrester, the percentage of runs finding the global optimum was highest for the proximity-based approach across exploitative, explorative, and adaptive $\beta$ settings; for example, under explorative $\beta=5$, the reported values were 42.3% for fidelity-weighted, 85.1% for MF-GPR-UCB, and 92.9% for proximity-based. On Bohachevsky, all methods had similar regret, with MF-GPR-UCB the most consistent overall and proximity underperforming in the exploitative regime. On Himmelblau, proximity-based achieved a good regret–high-fidelity-usage balance with tight variance, whereas MF-GPR-UCB had high variance in exploitative and adaptive modes. On Oregonator Hopf search, regret was similar overall; fidelity-weighted had the lowest high-fidelity usage, but proximity-based attained better regret with slightly higher high-fidelity usage, while MF-GPR-UCB used more high fidelity and struggled to reduce regret efficiently [2508.01013].

The chemical kinetics applications illustrate both heterogeneous and homogeneous fidelity relations. In the toy enzyme problem, the low fidelity is a QSSA-reduced model and the optimization target is to minimize $|X(E;T=10)-0.67|$. In the Oregonator problem, the low fidelity is also derived from QSSA reduction and the objective is to minimize $|\operatorname{Re}(\lambda(x^*,p))|$ over $p=(T,f)$. In dynamic ammonia catalysis, the parameters are $\nu \in [10^2,10^5]\ \text{Hz}$ and $\phi \in [0.5,0.99]$ for a periodic strain square wave between $\pm 4\%$; the low fidelity uses looser ODE tolerances, whereas the high fidelity uses a Newton–Krylov GMRES periodic steady-state solver. For this case, the proximity-based method achieved similar regret with significantly lower high-fidelity usage than MF-GPR-UCB and the fidelity-weighted baseline, and it also outperformed standard single-fidelity Bayesian optimization in high-fidelity count and final-regret distribution [2508.01013].

The limitations are correspondingly specific. The same source states that fidelity-weighted acquisition can fail through poor low-to-high information exchange and can get stuck near low-fidelity optima, as on Forrester. MF-GPR-UCB can exhibit inconsistent high-fidelity usage and regret, wider variance across runs, and occasionally high high-fidelity usage even at low $\Lambda$. The proximity-based method is presented as robust and simple, but the paper does not provide formal MF-GP-specific regret bounds for it or for MF-GPR-UCB [2508.01013].

More broadly, the earlier GP formulation notes a different limitation: it does not fit a joint multi-output GP and instead depends on explicit discrepancy bounds $\zeta^{(m)}$; if strong cross-fidelity correlations are available beyond such bounds, co-kriging-based models may exploit them better, though the theoretical guarantees in that earlier analysis hinge on the explicit bias-bound construction [1603.06288]. This suggests that the main tension in the MF-UCB literature lies between analytic tractability based on conservative discrepancy control and statistical efficiency based on richer cross-fidelity modeling.

Source: https://www.emergentmind.com/topics/multi-fidelity-upper-confidence-bound-ucb