Multi-Fidelity UCB Optimization

• Multi-Fidelity UCB is a cost-aware sequential decision framework that uses optimistic bounds to combine biased lower-fidelity approximations with expensive high-fidelity evaluations.
• It integrates uncertainty, approximation error, and evaluation cost through adaptive fidelity selection, thereby screening candidates with cheap evaluations before committing to costly ones.
• Applications span multi-armed bandit problems and Gaussian process optimization, balancing regret minimization with budget constraints while offering tunable performance trade-offs.

Searching arXiv for the cited MF-UCB and related multi-fidelity UCB papers to ground the article. Multi-Fidelity Upper Confidence Bound (MF-UCB) denotes a class of optimistic, cost-aware sequential decision procedures for optimization and bandit problems in which the target quantity is available at a highest fidelity, while cheaper lower fidelities provide biased or approximate information. Across its main formulations, MF-UCB combines confidence bounds with explicit fidelity selection rules so that low-cost evaluations screen large portions of the search space or arm set, and expensive high-fidelity evaluations are concentrated on candidates that remain plausible after lower-fidelity filtering. In the multi-armed bandit setting, this is formalized through bias-bounded approximations with fidelity-dependent costs (Kandasamy et al., 2016). In Gaussian-process bandit optimization, the same principle appears as fidelity-wise upper bounds on the true objective, aggregated into a single optimistic index (Kandasamy et al., 2016). More recent work has also implemented MF-UCB with multi-fidelity Gaussian processes of Kennedy–O’Hagan type, replacing separate single-fidelity posteriors with a co-kriging surrogate and adding tunable cost-ratio controls (Manoj et al., 1 Aug 2025).

1. Core concept and problem setting

The defining feature of MF-UCB is the joint treatment of three quantities: uncertainty, approximation error, and evaluation cost. The highest fidelity is the target of interest and is the most expensive to query. Lower fidelities are cheaper, but they are not exact substitutes; they are biased approximations or simplified models. MF-UCB uses upper confidence bounds to propagate lower-fidelity information into decisions about where or which arm to evaluate next, while explicitly accounting for fidelity-specific error and cost (Kandasamy et al., 2016).

In the stochastic multi-armed bandit formulation, there are $K$ arms and $M$ fidelities. Evaluating arm $k$ at fidelity $m$ incurs cost $\lambda^{(m)}$, with lower fidelities satisfying $\lambda^{(m)} < \lambda^{(M)}$. The highest fidelity has mean $\mu_k$, while lower fidelities have means $\mu_k^{(m)}$ that obey fidelity-wise bias bounds

$\bigl| \mu_k^{(m)} - \mu_k \bigr| \le \zeta^{(m)},$

with nondecreasing costs and nonincreasing bias bounds across fidelities (Kandasamy et al., 2016).

In Gaussian-process optimization, the objective is to maximize an expensive black-box function $f \equiv f^{(M)}$ on a domain $M$0, given access to lower-fidelity approximations $M$1. Here the discrepancy is expressed through uniform approximation bounds,

$M$2

and each fidelity is assigned a GP posterior used to form a valid upper bound on the true objective (Kandasamy et al., 2016).

A plausible implication is that MF-UCB is best understood not as a single algorithm but as a design pattern: optimism is retained, but the optimistic index is modified so that cheap, imperfect information can be exploited without treating it as exact.

2. Multi-armed bandit MF-UCB

In the multi-fidelity multi-armed bandit setting, MF-UCB extends classical UCB by constructing fidelity-specific upper bounds on the true arm mean. With empirical mean $M$3 and confidence radius $M$4, a canonical fidelity-wise index is

$M$5

The additive $M$6 term compensates for the worst-case bias of fidelity $M$7, so each $M$8 serves as an upper bound on the highest-fidelity mean $M$9 (Kandasamy et al., 2016).

Because every fidelity yields a valid upper bound, the arm-level optimistic index is aggregated by taking the tightest one:

$k$0

The arm selected at round $k$1 is the one with the largest aggregated upper bound,

$k$2

This “minimum across fidelities” is the central structural move in MF-UCB: low fidelities are used whenever they suffice to produce a tight certificate, and higher fidelities are invoked only when lower-fidelity uncertainty or bias becomes the limiting factor (Kandasamy et al., 2016).

Fidelity selection is governed by a promotion rule. The guiding intuition is that if the statistical uncertainty at fidelity $k$3 is already below the fidelity’s inherent bias, then additional samples at that fidelity cannot materially sharpen the bound. A concrete rule is to choose, for the selected arm, the smallest fidelity satisfying

$k$4

and otherwise promote to the highest fidelity. The synthesized description of the method states that MF-UCB uses lower fidelities to quickly eliminate suboptimal arms and reserves expensive experiments for a small set of promising candidates; it also states that the method is nearly optimal under certain conditions (Kandasamy et al., 2016).

The associated objective is budget-aware regret minimization under a total cost budget $k$5. Costs affect performance through the total number of possible plays and through the algorithm’s ability to substitute cheap lower-fidelity observations for expensive top-fidelity ones. This suggests that the principal advantage of bandit MF-UCB emerges when the lower fidelities are both substantially cheaper and sufficiently informative.

3. Gaussian-process MF-GP-UCB

MF-GP-UCB, introduced for Gaussian-process bandit optimization, translates the same principle into continuous black-box optimization. The setting assumes $k$6 fidelities, fidelity-dependent query costs $k$7, and approximation bounds $k$8 satisfying $k$9 (Kandasamy et al., 2016).

At each fidelity $m$0, a separate GP posterior is maintained using only the data collected at that fidelity. If $m$1 and $m$2 denote the posterior mean and standard deviation, the fidelity-wise upper confidence bound on the true objective is

$m$3

The approximation term $m$4 converts a confidence bound on $m$5 into a confidence bound on $m$6. The combined acquisition is then the pointwise minimum across fidelities,

$m$7

and the next query location is chosen by

$m$8

This retains the optimistic logic of GP-UCB while incorporating multiple biased approximations (Kandasamy et al., 2016).

Fidelity selection is handled by uncertainty thresholds $m$9. After choosing $\lambda^{(m)}$0, the method queries the smallest fidelity for which the scaled uncertainty remains sufficiently large:

$\lambda^{(m)}$1

The interpretation given in the source is that the algorithm should not over-invest at a low fidelity once its uncertainty is already smaller than the bias $\lambda^{(m)}$2; at that point escalation becomes appropriate (Kandasamy et al., 2016).

The theoretical analysis emphasizes “good sets” in which the top-fidelity queries concentrate. For $\lambda^{(m)}$3, the good set is defined as

$\lambda^{(m)}$4

with an inflated version $\lambda^{(m)}$5 incorporating the threshold $\lambda^{(m)}$6. The analysis states that almost all top-fidelity queries are confined to such sets, so the simple-regret bound depends on the maximum information gain over a reduced region rather than the entire domain. The resulting improvement is described as replacing $\lambda^{(m)}$7 with $\lambda^{(m)}$8 or a mild dilation thereof, while preserving the same $\lambda^{(m)}$9 rate (Kandasamy et al., 2016).

A common misconception is that MF-GP-UCB is simply co-kriging with UCB. The formulation summarized here explicitly states the opposite: it maintains a separate GP posterior for each fidelity and introduces cross-fidelity coupling only through the discrepancy bounds $\lambda^{(m)} < \lambda^{(M)}$0 rather than a joint multi-output GP (Kandasamy et al., 2016).

4. MF-UCB with multi-fidelity Gaussian processes

A later implementation modifies the MF-UCB idea by replacing separate per-fidelity GPs with a two-fidelity Kennedy–O’Hagan co-kriging model. In this formulation, the low fidelity is modeled as

$\lambda^{(m)} < \lambda^{(M)}$1

with $\lambda^{(m)} < \lambda^{(M)}$2 and squared exponential $\lambda^{(m)} < \lambda^{(M)}$3, while the discrepancy process is

$\lambda^{(m)} < \lambda^{(M)}$4

with $\lambda^{(m)} < \lambda^{(M)}$5 and squared exponential $\lambda^{(m)} < \lambda^{(M)}$6. The high fidelity is defined autoregressively by

$\lambda^{(m)} < \lambda^{(M)}$7

where $\lambda^{(m)} < \lambda^{(M)}$8 is a scalar scaling hyperparameter learned from data (Manoj et al., 1 Aug 2025).

Under mixed-fidelity observations, the posterior mean and variance at high fidelity are

$\lambda^{(m)} < \lambda^{(M)}$9

and

$\mu_k$0

Nested designs $\mu_k$1 are preferred; for non-nested designs, $\mu_k$2 is used to form the discrepancy target. The formulas are presented as essentially noise-free, with diagonal jitter added to the kernel matrices if noise handling is required (Manoj et al., 1 Aug 2025).

Within this MF-GP surrogate, the implemented MF-UCB acquisition is

$\mu_k$3

$\mu_k$4

with the local inter-fidelity error estimated by

$\mu_k$5

Candidate selection uses the combined bound

$\mu_k$6

and fidelity selection is determined through

$\mu_k$7

querying the low fidelity if $\mu_k$8 and otherwise the high fidelity (Manoj et al., 1 Aug 2025).

The implemented procedure, called MF-GPR-UCB, retrains the MF-GP at each iteration by optimizing kernel hyperparameters via marginal likelihood, uses L-BFGS multi-start to optimize the acquisition, and recommends reserving a small number of high-fidelity evaluations for final verification at the best $\mu_k$9 location. The same source explicitly states that it does not claim formal regret guarantees for MF-GPR-UCB with MF-GPs (Manoj et al., 1 Aug 2025).

This development marks an important methodological distinction. Earlier MF-GP-UCB uses independent per-fidelity GPs and analytic bias bounds $\mu_k^{(m)}$0 (Kandasamy et al., 2016), whereas the newer MF-GPR-UCB uses an explicit inter-fidelity correlation model and estimates $\mu_k^{(m)}$1 pointwise from the discrepancy between MF-GP means (Manoj et al., 1 Aug 2025).

5. Tunability, cost ratios, and alternative acquisition design

A prominent practical theme in recent multi-fidelity UCB work is tunability through cost ratios. In the MF-GPR-UCB formulation, the tunable parameters are $\mu_k^{(m)}$2, $\mu_k^{(m)}$3, and $\mu_k^{(m)}$4. Fixed $\mu_k^{(m)}$5 choices are divided into exploitative settings $\mu_k^{(m)}$6 and explorative settings $\mu_k^{(m)}$7, while an adaptive schedule borrowed from earlier work is

$\mu_k^{(m)}$8

The cost ratio $\mu_k^{(m)}$9 controls high-fidelity usage through the threshold $\bigl| \mu_k^{(m)} - \mu_k \bigr| \le \zeta^{(m)},$0 (Manoj et al., 1 Aug 2025).

The same study introduces a proximity-based acquisition strategy as an alternative to per-fidelity acquisition functions. It selects the next point using only the high-fidelity acquisition,

$\bigl| \mu_k^{(m)} - \mu_k \bigr| \le \zeta^{(m)},$1

and then chooses the fidelity from the minimum distance to the existing low-fidelity design,

$\bigl| \mu_k^{(m)} - \mu_k \bigr| \le \zeta^{(m)},$2

with proximity radius

$\bigl| \mu_k^{(m)} - \mu_k \bigr| \le \zeta^{(m)},$3

If $\bigl| \mu_k^{(m)} - \mu_k \bigr| \le \zeta^{(m)},$4, the algorithm queries the low fidelity; otherwise it queries the high fidelity. The stated goal is to use a single acquisition function and select the fidelity by a proximity criterion that depends on local low-fidelity sample density and a tunable cost ratio (Manoj et al., 1 Aug 2025).

The paper also evaluates a fidelity-weighted baseline. For counts $\bigl| \mu_k^{(m)} - \mu_k \bigr| \le \zeta^{(m)},$5 and $\bigl| \mu_k^{(m)} - \mu_k \bigr| \le \zeta^{(m)},$6 of low- and high-fidelity evaluations, the cost penalties are

$\bigl| \mu_k^{(m)} - \mu_k \bigr| \le \zeta^{(m)},$7

and the modified acquisitions are

$\bigl| \mu_k^{(m)} - \mu_k \bigr| \le \zeta^{(m)},$8

$\bigl| \mu_k^{(m)} - \mu_k \bigr| \le \zeta^{(m)},$9

The candidate is selected by maximizing the larger of the two penalized acquisitions, and the fidelity is then chosen by whichever penalized acquisition is larger at that candidate (Manoj et al., 1 Aug 2025).

The reported comparison gives three distinct behaviors. The proximity-based strategy is described as providing smooth, predictable high-fidelity-usage control across $f \equiv f^{(M)}$0 with tight distributions. MF-GPR-UCB shows a smoother average trend than the fidelity-weighted approach, but also wide variance and sometimes elevated high-fidelity usage even at low $f \equiv f^{(M)}$1. The fidelity-weighted approach is described as sharply sensitive to the cost ratio, with abrupt jumps in high-fidelity usage and poor low-to-high information exchange (Manoj et al., 1 Aug 2025).

A plausible implication is that, in practice, MF-UCB variants differ at least as much in their fidelity-allocation mechanism as in their optimistic index.

6. Empirical behavior, applications, and limitations

The recent benchmark study compares MF-GPR-UCB, the proximity-based strategy, and the fidelity-weighted approach on Forrester, Bohachevsky, toy enzyme, Himmelblau, Oregonator Hopf search, and dynamic ammonia catalysis. Its overall summary is that proximity-based acquisition consistently provided predictable control of high-fidelity usage with competitive regret, often the lowest high-fidelity fraction used; MF-GPR-UCB showed good balance in some cases but inconsistency in others; and the fidelity-weighted approach tended to rely heavily on high fidelity and was very sensitive to the cost ratio (Manoj et al., 1 Aug 2025).

Several task-specific outcomes are reported. On Forrester, the percentage of runs finding the global optimum was highest for the proximity-based approach across exploitative, explorative, and adaptive $f \equiv f^{(M)}$2 settings; for example, under explorative $f \equiv f^{(M)}$3, the reported values were 42.3% for fidelity-weighted, 85.1% for MF-GPR-UCB, and 92.9% for proximity-based. On Bohachevsky, all methods had similar regret, with MF-GPR-UCB the most consistent overall and proximity underperforming in the exploitative regime. On Himmelblau, proximity-based achieved a good regret–high-fidelity-usage balance with tight variance, whereas MF-GPR-UCB had high variance in exploitative and adaptive modes. On Oregonator Hopf search, regret was similar overall; fidelity-weighted had the lowest high-fidelity usage, but proximity-based attained better regret with slightly higher high-fidelity usage, while MF-GPR-UCB used more high fidelity and struggled to reduce regret efficiently (Manoj et al., 1 Aug 2025).

The chemical kinetics applications illustrate both heterogeneous and homogeneous fidelity relations. In the toy enzyme problem, the low fidelity is a QSSA-reduced model and the optimization target is to minimize $f \equiv f^{(M)}$4. In the Oregonator problem, the low fidelity is also derived from QSSA reduction and the objective is to minimize $f \equiv f^{(M)}$5 over $f \equiv f^{(M)}$6. In dynamic ammonia catalysis, the parameters are $f \equiv f^{(M)}$7 and $f \equiv f^{(M)}$8 for a periodic strain square wave between $f \equiv f^{(M)}$9; the low fidelity uses looser ODE tolerances, whereas the high fidelity uses a Newton–Krylov GMRES periodic steady-state solver. For this case, the proximity-based method achieved similar regret with significantly lower high-fidelity usage than MF-GPR-UCB and the fidelity-weighted baseline, and it also outperformed standard single-fidelity Bayesian optimization in high-fidelity count and final-regret distribution (Manoj et al., 1 Aug 2025).

The limitations are correspondingly specific. The same source states that fidelity-weighted acquisition can fail through poor low-to-high information exchange and can get stuck near low-fidelity optima, as on Forrester. MF-GPR-UCB can exhibit inconsistent high-fidelity usage and regret, wider variance across runs, and occasionally high high-fidelity usage even at low $M$00. The proximity-based method is presented as robust and simple, but the paper does not provide formal MF-GP-specific regret bounds for it or for MF-GPR-UCB (Manoj et al., 1 Aug 2025).

More broadly, the earlier GP formulation notes a different limitation: it does not fit a joint multi-output GP and instead depends on explicit discrepancy bounds $M$01; if strong cross-fidelity correlations are available beyond such bounds, co-kriging-based models may exploit them better, though the theoretical guarantees in that earlier analysis hinge on the explicit bias-bound construction (Kandasamy et al., 2016). This suggests that the main tension in the MF-UCB literature lies between analytic tractability based on conservative discrepancy control and statistical efficiency based on richer cross-fidelity modeling.