Multi-Fidelity Batch Bayesian Optimization

• Multi-Fidelity Batch Bayesian Optimization is a technique that selects both evaluation locations and fidelity levels to efficiently balance cost and accuracy using structured surrogate models.
• It leverages batch and asynchronous strategies to conduct multiple evaluations simultaneously, reducing overall computational expense while targeting high-fidelity outcomes.
• Acquisition functions in MFBO integrate metrics like information gain per unit cost and redundancy reduction, enabling practical and effective resource allocation in diverse applications.

Multi-Fidelity Batch Bayesian Optimization is the class of Bayesian optimization methods in which the optimizer chooses not only where to evaluate an expensive black-box objective, but also at which fidelity and, in batch variants, which set of evaluations to launch jointly. The target is typically the highest-fidelity function, while lower fidelities provide cheaper but less accurate information; batch and asynchronous variants further exploit parallel resources or staggered evaluation times. Across the literature, the central design problem is to balance information gain, fidelity-dependent cost, and redundancy among simultaneously selected queries, under surrogate models that transfer information across fidelities (Zhang et al., 2024, Li et al., 2021, Folch et al., 2022).

1. Formal problem setting

A standard starting point is the expensive-objective problem

$\min_{x\in \mathbb{A}} f(x), \qquad f(x):=g(x,1),$

where $g(x,s)$ is a fidelity-dependent approximation and $s\in[0,1]^m$ denotes one or more continuous fidelity controls rescaled so that $s=1$ is the highest fidelity and $s=0$ the lowest (Wu et al., 2019). In discrete multi-fidelity formulations, one instead has correlated information sources

$f_1,\dots,f_{m-1},f_m,$

with $f_m$ the target fidelity, query actions $(x,\ell)$, costs $\lambda_\ell$, and noisy observations

$$y_{x,\ell}=f_\ell(x)+\epsilon,\qquad \epsilon\sim \mathcal N(0,\sigma^2).$$

A budgeted objective studied in the Gaussian-process framework is to choose actions adaptively so that the cumulative reward from highest-fidelity queries is maximized subject to a total cost budget $g(x,s)$0 (Song et al., 2018).

Batch formulations promote joint selection of multiple fidelity-input pairs. BMBO-DARN explicitly frames the batch decision as selecting

$g(x,s)$1

so as to maximize information about the optimum while accounting for query costs $g(x,s)$2 (Li et al., 2021). In practical domains, the decision variables can also include fidelity as part of the input. A CFD burner-design formulation uses the design vector

$g(x,s)$3

where mesh element size $g(x,s)$4 induces a continuous fidelity index $g(x,s)$5, while the optimization seeks to maximize mean reactor temperature subject to an emissions cap $g(x,s)$6 and box constraints on $g(x,s)$7, $g(x,s)$8, and $g(x,s)$9 (Lima et al., 28 Nov 2025). A bioprocess formulation likewise optimizes process variables and categorical clone choice across discrete scales—microtiter plate, microbioreactor, and pilot scale—under explicit experiment costs and batched execution (Martens et al., 14 Aug 2025).

This diversity of formulations suggests that “fidelity” is not restricted to simulator labels. In the cited works it appears as a discrete level, a continuous control, a mesh-resolution variable, a training-budget vector, or an experimental scale, but the underlying question is the same: how to spend budget on cheap approximations and expensive target evaluations so that the highest-fidelity optimum is found efficiently.

2. Fidelity structure and surrogate modelling

The surrogate model is the mechanism by which low-fidelity observations become useful for the target task. A general perspective on multi-fidelity Bayesian optimization emphasizes that low-fidelity information is embedded in the prior through structured multi-fidelity surrogates, rather than appended as unrelated side data (Zhang et al., 2024). In Gaussian-process language, one common prior is the linear model of coregionalization,

$s\in[0,1]^m$0

with autoregressive special cases such as

$s\in[0,1]^m$1

and recursive variants that propagate the posterior of the previous fidelity level forward (Zhang et al., 2024).

A different Gaussian-process construction appears in MF-MI-Greedy, which assumes a joint GP model over fidelities and, in experiments, uses the additive representation

$s\in[0,1]^m$2

where $s\in[0,1]^m$3 is the shared target component and $s\in[0,1]^m$4 is a fidelity-specific discrepancy GP. In that representation, every fidelity shares the latent target structure $s\in[0,1]^m$5, so a low-fidelity query can reduce uncertainty at the target fidelity (Song et al., 2018).

Hyperparameter-tuning work adopts a GP over the augmented input $s\in[0,1]^m$6, where $s\in[0,1]^m$7 may include multiple continuous fidelity controls and trace structure. The key object is the fidelity-dependent objective $s\in[0,1]^m$8 together with the set $s\in[0,1]^m$9 of fidelities that are automatically observed when evaluating at $s=1$0, such as earlier iteration counts along a training trace (Wu et al., 2019). This is a different notion of coupling: one evaluation can reveal a sequence of lower-fidelity values at the same design point.

Neural surrogates are used when the correlation across fidelities is strong but structurally complex. DNN-MFBO models fidelity $s=1$1 as a nonlinear transformation of the previous fidelity,

$s=1$2

with uncertainty represented through a Bayesian last layer (Li et al., 2020). BMBO-DARN strengthens this dependency into a fully auto-regressive chain,

$s=1$3

implemented as a set of Bayesian neural networks and fitted with Hamiltonian Monte Carlo (Li et al., 2021).

Non-hierarchical settings motivate yet another surrogate class. MuFASa uses a single Latent Variable Gaussian Process in which fidelity levels are treated as qualitative categories mapped to latent coordinates, and the GP correlation is defined on the augmented space of quantitative inputs and latent fidelity embeddings (Chen et al., 2023). This avoids assuming a predefined hierarchy and instead learns inter-fidelity similarity from data.

Several recent engineering applications treat fidelity as an explicit input coordinate rather than a separate multi-level hierarchy. The burner-design framework places the mesh-driven fidelity variable directly in the GP input space and uses separate independent GPs for $s=1$4 and $s=1$5 in BoTorch and GPyTorch (Lima et al., 28 Nov 2025). The bioprocess framework similarly uses a GP over process variables, fidelity, and clone identity, with clone structure represented either by an Intrinsic Coregionalization Model or by an entity embedding kernel; the reported best-performing categorical representation is a 3D entity embedding (Martens et al., 14 Aug 2025).

3. Acquisition design and cost-sensitive utility

Acquisition design is the second defining component of multi-fidelity batch BO. A general perspective paper decomposes BO and MFBO into a prior $s=1$6, an optimization policy $s=1$7, and a global maximization program $s=1$8, while noting that many multi-fidelity acquisition functions choose location and fidelity jointly via $s=1$9 or choose location first and fidelity later via $s=0$0 (Zhang et al., 2024). The same source also notes that many existing cost-aware or fidelity-aware rules are heuristic.

A principled Gaussian-process alternative is MF-MI-Greedy, whose exploration phase selects low-fidelity queries according to mutual information about the target fidelity per unit cost:

$s=0$1

The method alternates a low-fidelity exploration phase with a target-fidelity optimization phase, and the Explore-LF subroutine greedily chooses

$s=0$2

then stops when budget is exhausted, when target fidelity is better than all low fidelities, or when the cumulative benefit-cost ratio becomes too small (Song et al., 2018).

The trace-aware knowledge-gradient adopts a value-of-information formulation. Its basic acquisition is

$s=0$3

where $s=0$4 is the expected loss after one more trace observation and the numerator is the value of information. A second, “0-avoiding” variant modifies the value of information so that fidelities with zero components receive zero value, thereby preventing the optimizer from repeatedly selecting vanishingly cheap but nearly useless points (Wu et al., 2019).

In explicitly batched settings, BMBO-DARN extends Max-value Entropy Search to batches of fidelity-input pairs:

$s=0$5

The numerator is the mutual information between the entire batch and the optimum, so highly correlated batch points reduce joint entropy and therefore reduce the acquisition value. This is the paper’s formal mechanism for encouraging diversity without a separate combinatorial redundancy penalty (Li et al., 2021).

Multi-fidelity batch acquisition can also be expressed through extensions of standard batch BO criteria. In the bioprocess framework, the cost-aware multi-fidelity batch expected-improvement variant is

$s=0$6

where $s=0$7 is the same batch projected to target fidelity and $s=0$8 is the correlation between the batch at current fidelity and its target-fidelity projection. The same work also uses extended qUCB and GIBBON, the latter described as information-theoretic and naturally batch-capable and multi-fidelity-capable (Martens et al., 14 Aug 2025).

In constrained engineering problems, the acquisition may incorporate feasibility and runtime. The burner-design framework uses constrained noisy expected improvement,

$s=0$9

and then optimizes a penalized objective

$f_1,\dots,f_{m-1},f_m,$0

so that expected improvement, feasibility, fidelity, and runtime are all present in the decision rule (Lima et al., 28 Nov 2025).

A plausible implication is that acquisition design in this area is increasingly less about a single generic exploration–exploitation score and more about composing information, cost, fidelity, feasibility, and parallel redundancy into a single utility.

4. Batch construction, parallelism, and asynchrony

The literature distinguishes explicit batch algorithms from sequential methods that only have a batch-like effect. MF-MI-Greedy is not a pure batch algorithm, but it does select multiple low-fidelity points before a target-fidelity point, which the authors describe as batch-like in spirit because posterior updates after a grouped set of low-fidelity evaluations influence the next high-fidelity decision (Song et al., 2018). By contrast, taKG and BMBO-DARN both define genuine joint batch criteria (Wu et al., 2019, Li et al., 2021).

A representative cross-section of methods is summarized below.

Method	Core mechanism	Batch relevance
MF-MI-Greedy	Cost-sensitive mutual information over a joint GP	Structured batch-like exploration (Song et al., 2018)
taKG	Trace-aware knowledge-gradient over $f_1,\dots,f_{m-1},f_m,$1	Batch and derivative variants (Wu et al., 2019)
BMBO-DARN	Batch MES with fully auto-regressive BNN surrogate	Joint batch selection over fidelity-input pairs (Li et al., 2021)
Asynchronous MF batch framework	Pending-point-aware acquisition plus fidelity rule	Asynchronous batch proposals (Folch et al., 2022)
Bioprocess MF batch BO	Extended qLogEI, qUCB, and GIBBON over fidelity batches	Explicit per-fidelity batch scoring (Martens et al., 14 Aug 2025)

BMBO-DARN is notable for how it avoids combinatorial search over fidelity assignments. Rather than enumerating all $f_1,\dots,f_{m-1},f_m,$2 combinations, it performs coordinate-wise alternating optimization: with all other batch elements fixed, it updates one pair $f_1,\dots,f_{m-1},f_m,$3 at a time and guarantees that each coordinate update does not decrease the batch acquisition. The reported time complexity is $f_1,\dots,f_{m-1},f_m,$4, linear in the number of fidelities and batch size rather than exponential (Li et al., 2021).

Asynchronous multi-fidelity batch BO introduces another layer of realism. In that setting, the optimizer tracks a pending set

$f_1,\dots,f_{m-1},f_m,$5

of unresolved evaluations and must choose new experiments before all prior ones finish. A general framework combines a multi-fidelity surrogate, a highest-fidelity batch acquisition for selecting $f_1,\dots,f_{m-1},f_m,$6, and a separate fidelity-selection rule. The asynchronous component can be implemented with Thompson sampling, fantasies, or local penalization, while fidelity is chosen either by a variance/UCB-based rule or by an information-based rule inspired by MF-MES (Folch et al., 2022).

The bioprocess workflow adds domain-specific batching constraints. For qEI and qUCB it uses fantasizing, Kriging Believer, or Constant Liar; for large low-fidelity microtiter-plate batches it performs a grid search over $f_1,\dots,f_{m-1},f_m,$7, samples clones using Latin hypercube sampling, evaluates candidate batches with GIBBON, and then chooses the best batch. Higher fidelities use multi-start L-BFGS-B and fantasy-based greedy batch construction (Martens et al., 14 Aug 2025). This is a reminder that practical batch BO often departs from a single closed-form joint optimizer once mixed variables, laboratory constraints, or large low-fidelity batch sizes are introduced.

5. Guarantees, assumptions, and failure modes

Formal guarantees in multi-fidelity BO are strongest in the sequential GP setting. MF-MI-Greedy introduces an episode-based regret notion in which an episode consists of several low-fidelity actions followed by one target-fidelity action. If an episode has total cost $f_1,\dots,f_{m-1},f_m,$8 and ends at $f_1,\dots,f_{m-1},f_m,$9, the episode regret is

$f_m$0

and for total budget $f_m$1 the cumulative regret is

$f_m$2

Under a target-fidelity optimizer with regret bounded by $f_m$3 and a stopping threshold satisfying $f_m$4, the method is asymptotically no-regret in the sense that

$f_m$5

with high probability (Song et al., 2018).

Theoretical analysis also exists at the acquisition-optimization level. For taKG, the paper derives an unbiased stochastic gradient estimator and proves that, with diminishing step sizes satisfying

$f_m$6

stochastic gradient ascent converges almost surely to a connected set of stationary points of the acquisition function (Wu et al., 2019). This is not a global optimization guarantee for the full BO loop, but it is a substantive result because the acquisition itself is simulation-defined.

By contrast, broader surveys emphasize that convergence guarantees are abundant for generic BO but lacking for MFBO, especially once one considers heuristic cost/fidelity corrections, difficult acquisition landscapes, or batch selection (Zhang et al., 2024). The same perspective explicitly identifies lack of principled acquisition design, difficulty of acquisition optimization, and limited convergence theory as open problems for MFBO.

Empirical chemistry work underscores a common misconception: multi-fidelity optimization is not automatically better than single-fidelity BO. In that study, MFBO improved over SFBO in only 2 of 4 problem datasets, and the practical guidance is that lower-fidelity data helps only when it is very strongly correlated with the high-fidelity objective, typically above $f_m$7, and much cheaper, often $f_m$8–$f_m$9 lower cost than high fidelity (Judge et al., 2024). That paper also recommends the CRHF metric because standard regret does not account naturally for cheap low-fidelity evaluations.

These results suggest a layered picture of reliability. Strong guarantees are available when the model class, fidelity structure, and stopping rules are carefully controlled; outside that regime, success depends heavily on whether the chosen surrogate and acquisition actually reflect the cross-fidelity correlations and costs present in the application.

6. Applications, empirical behavior, and current directions

The application range is broad. In hyperparameter tuning of iteratively trained models, taKG is designed for continuous fidelities such as training set size and number of training iterations, and its batch version with batch size $(x,\ell)$0 substantially improves over batch KG, batch EI, and Hyperband on MNIST; the paper also reports final test classification errors after full retraining of about $(x,\ell)$1 on CIFAR-10 and $(x,\ell)$2 on SVHN (Wu et al., 2019). BMBO-DARN evaluates batch multi-fidelity hyperparameter optimization on CNNs, online LDA, XGBoost, and PINNs, and reports the best performance versus accumulated cost across all four applications (Li et al., 2021).

In engineering design, the burner-design framework uses continuous mesh fidelity together with a runtime surrogate and reports comparable convergence to a hypothetical single-fidelity campaign with about $(x,\ell)$3 percent lower total wall time; low-fidelity evaluations with $(x,\ell)$4 are reported to be roughly an order of magnitude faster than full-fidelity ones (Lima et al., 28 Nov 2025). DNN-MFBO, though not itself a batch method, reports lower simple regret and lower inference regret on synthetic and engineering tasks, and acquisition computation about $(x,\ell)$5 faster than MF-MES and $(x,\ell)$6 faster than MF-PES on average, which is directly relevant because batch BO often requires repeated acquisition evaluations (Li et al., 2020).

Laboratory and industrial settings have driven asynchronous formulations. In the battery-design application, high fidelity corresponds to pouch-cell testing and low fidelity to coin-cell testing with accelerated procedures; low-fidelity cost is set to $(x,\ell)$7, high-fidelity cost to $(x,\ell)$8, and high fidelity takes about $(x,\ell)$9 times longer than low fidelity. The reported outcome is that the combined asynchronous multi-fidelity batch methods outperform plain UCB, MF-GP-UCB without batching, and single-fidelity local penalization, with both batch-aware multi-fidelity methods emerging as the strongest performers (Folch et al., 2022).

Bioprocess development provides an explicit multi-fidelity batch workflow across scales. In Distribution A, GIBBON reaches an average objective of $\lambda_\ell$0 mg/L at a cost of €41,164, about $\lambda_\ell$1 cheaper than industrial DoE, while the best DoE baseline reaches $\lambda_\ell$2 mg/L; in the same scenario qUCB performs similarly to DoE in final value but at about $\lambda_\ell$3 lower cost than LHS. In Distribution B, the reported terminal values are $\lambda_\ell$4 mg/L for GIBBON, $\lambda_\ell$5 mg/L for qUCB, $\lambda_\ell$6 mg/L for qLogEI, and $\lambda_\ell$7 mg/L for the best DoE baseline, with qUCB and qLogEI delivering about $\lambda_\ell$8 and $\lambda_\ell$9 cost reduction, respectively (Martens et al., 14 Aug 2025).

Across domains, several empirical regularities recur. Early iterations often rely on cheaper fidelities for exploration, later iterations shift toward higher fidelity near promising regions, and acquisition choice materially changes performance. At the same time, the chemistry perspective is a caution against overgeneralization: MFBO does not consistently outperform SFBO, and the gain depends sharply on fidelity correlation, cost ratio, dimensionality, and acquisition choice (Judge et al., 2024).

Current directions in the cited works include more principled acquisition design from Bayesian decision theory, tighter integration of scientific structure into the prior, stronger use of continuous-fidelity representations, and extensions of sequential MFBO ideas to explicit batch or asynchronous settings (Zhang et al., 2024, Lima et al., 28 Nov 2025). A plausible implication is that the mature form of multi-fidelity batch BO will be neither purely GP-based nor purely information-theoretic, but a family of methods that adapt surrogate class, fidelity representation, and batch policy to the geometry, physics, and scheduling constraints of the underlying problem.