Multi-Fidelity Active Learning (MFAL)

• Multi-Fidelity Active Learning (MFAL) is a framework that actively selects among models of varying cost and accuracy to optimize query value under strict budget constraints.
• It leverages surrogate models, cross-fidelity coupling, and cost-aware acquisition functions to efficiently balance information gain against computational expense.
• MFAL has been successfully applied to PDE surrogate modeling, rare-event simulation, and drug design, achieving notable efficiency improvements over single-fidelity approaches.

Multi-Fidelity Active Learning (MFAL) is a class of methods that combines multiple information sources with different cost–accuracy trade-offs and actively selects both the input and the fidelity of the next query under a limited budget. In the literature, the target object can be a scalar objective, a binary event indicator, a high-dimensional field, a molecular score, or the solution of a differential equation; the common structure is a family of fidelities with increasing accuracy and computational cost, together with a sequential or batch policy for deciding which $(x,\ell)$ or $(x,m)$ pairs to evaluate next (Pellegrini et al., 2022, Li et al., 2020, Hernandez-Garcia et al., 2023).

1. Problem formulations and scope

A standard MFAL formulation assumes a finite set of fidelities $L=\{1,\dots,F\}$ or $m=1,\dots,M$, with per-query costs $c_l$ or $\lambda_m$, and a target highest-fidelity map $f_H \equiv f_M$. The learning problem is budgeted: batch or sequential selections must satisfy constraints such as $\sum_{(l,x)\in S} c_l \le B$, $\sum_{t=1}^{T} c_{f_t}\le B$, or a total evaluation budget $\Lambda$ (Li et al., 2022, Nguyen et al., 2021, Hernandez-Garcia et al., 2023). In this sense, MFAL is broader than multi-fidelity surrogate modeling alone: it adds an acquisition rule that decides where and at which fidelity to spend the next unit of budget.

The outputs treated in MFAL are heterogeneous. In design optimization with noisy simulations, the objective is a scalar performance metric $(x,m)$0 evaluated by several models, solvers, or discretizations (Pellegrini et al., 2022). In deep surrogate learning, the target is a high-dimensional output $(x,m)$1, such as a PDE solution field, a topology optimization layout, or a CFD state (Li et al., 2020, Li et al., 2022). In rare-event simulation, the quantity of interest is the probability of failure defined by a limit-state function $(x,m)$2 (Chaudhuri et al., 2019, Chakroborty et al., 2022, Dhulipala et al., 2021). In discrete scientific discovery, the objective is to discover high-scoring candidates in a large structured design space while choosing among multiple oracle fidelities (Hernandez-Garcia et al., 2023). In active search, the utility is the number of high-fidelity positives found before the budget is exhausted (Nguyen et al., 2021).

A representative mathematical form for cost-aware MFAL in black-box discovery is

$(x,m)$3

where $(x,m)$4 is the maximum of the highest-fidelity objective, $(x,m)$5 is the observation at fidelity $(x,m)$6, and $(x,m)$7 is the current dataset (Hernandez-Garcia et al., 2023). Other formulations target the highest-fidelity output field, a latent representation of the highest-fidelity function, or the failure boundary rather than an optimum (Wu et al., 2023, Cutforth et al., 9 Oct 2025, Chaudhuri et al., 2019).

2. Surrogate models and cross-fidelity coupling

A central issue in MFAL is how to encode dependence across fidelities. A widely used construction is autoregressive co-kriging in the Kennedy–O’Hagan style, where the high-fidelity process is expressed as a scaled low-fidelity process plus a discrepancy term. In operator-informed Gaussian-process methods for differential equations, the solution is modeled as

$(x,m)$8

with independent Gaussian-process priors on $(x,m)$9 and $L=\{1,\dots,F\}$0, and the forcing inherits a corresponding autoregressive structure through the linear operator $L=\{1,\dots,F\}$1 (Raissi et al., 2016). In GP classification with binary outputs, an analogous latent formulation is

$L=\{1,\dots,F\}$2

with $L=\{1,\dots,F\}$3 and $L=\{1,\dots,F\}$4 independent GPs, so that low-fidelity information transfers to the high-fidelity classifier through $L=\{1,\dots,F\}$5 and the shared covariance structure (Cutforth et al., 9 Oct 2025).

Other papers depart from additive autoregression. The Recursive Non-Additive (RNA) emulator replaces the additive link $L=\{1,\dots,F\}$6 with the recursive form $L=\{1,\dots,F\}$7, where $L=\{1,\dots,F\}$8 is a Gaussian process on augmented inputs $L=\{1,\dots,F\}$9. This construction preserves closed-form posterior mean and variance under common kernels while allowing nonlinear, non-additive fidelity relationships (Heo et al., 2023). A different direction appears in correction-based methods, where the overall multi-fidelity prediction is a low-fidelity trained surrogate corrected with surrogates of the errors between consecutive fidelity levels. In the stochastic radial basis function approach for noisy design optimization, the paper defines

$m=1,\dots,M$0

and forms correction targets using the denoised surrogate prediction at the next level rather than raw low-fidelity simulations (Pellegrini et al., 2022).

High-dimensional-output settings motivate deep models. DMFAL uses a deep neural multi-fidelity model in which the latent output of fidelity $m=1,\dots,M$1 is concatenated to the original input and passed into fidelity $m=1,\dots,M$2, with linear projections from low-dimensional latent spaces to very high-dimensional outputs (Li et al., 2020). D-MFDAL instead argues that hierarchical hidden-feature passing can propagate low-fidelity errors upward and therefore learns, at each fidelity, disentangled local and global representations; the latent variables $m=1,\dots,M$3 at different fidelities are conditionally independent given those representations, and global sharing occurs through Bayesian aggregation in latent space rather than through sequential hidden-feature inheritance (Wu et al., 2023). MRA-FNO transfers the same idea to operator learning by introducing a probabilistic multi-resolution Fourier Neural Operator with resolution-aware embeddings and a deep-ensemble posterior approximation (Li et al., 2023).

Feature-augmented transfer models provide another coupling mechanism. In active airfoil optimization, the high-fidelity quantity is predicted by a Gaussian process whose input is the concatenation of the geometric design vector and the corresponding low-fidelity XFOIL output, rather than by an explicit autoregressive co-kriging relation (Robledo et al., 17 Mar 2026). In rare-event simulation with multiple low-fidelity models, each low-fidelity model is first corrected by an independent GP discrepancy model, and then local model probabilities are used to average or select among the corrected low-fidelity surrogates (Chakroborty et al., 2022).

3. Acquisition functions and fidelity policies

MFAL methods differ most visibly in their acquisition rules. One family uses posterior variance directly. In operator-informed GP inference for linear differential equations, the next high-fidelity observation of the forcing is selected by

$m=1,\dots,M$4

where $m=1,\dots,M$5 is the posterior variance of the forcing. The paper reports that posterior variance acts as an a-posteriori error indicator and that this simple criterion performs comparably to more involved information-theoretic ones for the tested cases (Raissi et al., 2016).

A second family uses mutual information or entropy reduction. DMFAL defines a cost-normalized mutual information $m=1,\dots,M$6 between a candidate query at fidelity $m=1,\dots,M$7 and the high-fidelity output, and computes the necessary entropies through multivariate Delta’s method, moment matching, and the Weinstein–Aronszajn identity (Li et al., 2020). BMFAL-BC extends this logic to batches by maximizing the expected mutual information between a batch of multi-fidelity queries and the high-fidelity function under a hard budget (Li et al., 2022). D-MFDAL introduces Multi-Fidelity Latent Information Gain,

$m=1,\dots,M$8

which targets expected information gain about the highest-fidelity latent variable $m=1,\dots,M$9 per unit cost (Wu et al., 2023). In binary simulation output, BPMI approximates mutual information directly in the Bernoulli-parameter space by a first-order Taylor expansion of the link function, thereby focusing acquisition near the classification boundary where the link derivative is large (Cutforth et al., 9 Oct 2025).

Task-specific policies also appear. In multi-fidelity active learning with GFlowNets, the acquisition is MF-MES, a cost-aware variant of Max-value Entropy Search,

$c_l$0

and the GFlowNet is trained on a reward derived from this acquisition so that sampling remains diverse over high-value modes (Hernandez-Garcia et al., 2023). RAAL uses Multifidelity Expected Improvement,

$c_l$1

with $c_l$2, and then solves a resource-aware mixed-integer linear program to choose a parallel batch under CPU-capacity constraints (Grassi et al., 2020).

Rare-event simulation favors adequacy tests rather than global-optimum acquisitions. mfEGRA first selects a location by the Expected Feasibility Function and then chooses the fidelity by cost-normalized, weighted one-step lookahead information gain toward the failure boundary (Chaudhuri et al., 2019). A related framework fuses a low-fidelity prediction with a GP correction and calls the high-fidelity model only when a subset-dependent U-function falls below a threshold; the dynamic threshold is tied to the current subset level and was introduced to avoid failure of static U-functions at very small failure probabilities (Dhulipala et al., 2021). In optimization-embedded airfoil MFAL, fidelity escalation is triggered by the coefficient of variation $c_l$3; if $c_l$4, a high-fidelity RANS call is made and the GP is retrained online (Robledo et al., 17 Mar 2026). In MF-LAL for drug design, fidelity promotion is threshold-based as well: the method moves from fidelity $c_l$5 to $c_l$6 when the surrogate uncertainty at the current query falls below a prescribed $c_l$7 (Eckmann et al., 2024).

4. Batch selection, diversity, and resource management

Sequential single-point querying is only one regime of MFAL. BMFAL-BC formulates batch selection as a budgeted maximization of expected mutual information and uses a weighted greedy algorithm that sequentially identifies each $c_l$8 pair while achieving a near $c_l$9-approximation under the knapsack-style budget (Li et al., 2022). Because the batch objective is a log-determinant mutual information, highly correlated queries provide diminishing returns, so the acquisition naturally discourages redundancy and promotes diversity within the batch (Li et al., 2022).

Diversity is explicit in discrete discovery settings. MF-GFN jointly samples $\lambda_m$0 rather than following a strict low$\lambda_m$1high pipeline, then samples $\lambda_m$2 candidate–fidelity pairs from the learned policy and keeps the top $\lambda_m$3 according to the acquisition. The paper argues that GFlowNets sample proportionally to reward across the discrete space and therefore cover multiple informative modes rather than collapsing to a single solution, unlike RL-based alternatives that frequently collapsed to single modes in complex domains (Hernandez-Garcia et al., 2023). In active search, MF-ENS is nonmyopic and budget aware: each expensive high-fidelity query allows several cheaper low-fidelity queries to complete while the high-fidelity result is pending, and the policy uses a rollout approximation to balance exploratory low-fidelity queries against the expected value of the remaining high-fidelity batch (Nguyen et al., 2021).

Resource awareness is most explicit in RAAL. After evaluating an acquisition value for each candidate–fidelity pair, RAAL solves a mixed-integer linear program whose objective combines total acquisition value with total resource utilization, while constraints enforce per-CPU capacity limits, one-sample-per-bin diversity, and fidelity consistency with the recursive GP model (Grassi et al., 2020). This makes batch construction a scheduling problem as well as an inference problem. A plausible implication is that MFAL is not only about information efficiency but also about matching acquisition logic to the execution model of the underlying compute platform.

5. Application domains and reported empirical behavior

The literature applies MFAL to inverse differential equations, surrogate modeling of PDE solution fields, reliability analysis, global design optimization, operator learning, active discovery in discrete spaces, drug design, aerodynamic optimization, and laser direct drive implosions (Raissi et al., 2016, Wu et al., 2023, Chaudhuri et al., 2019, Pellegrini et al., 2022, Hernandez-Garcia et al., 2023, Eckmann et al., 2024, Robledo et al., 17 Mar 2026, Crilly et al., 28 Aug 2025).

Domain	Reported outcome	Papers
Linear differential equations	“Fast convergence in relative error for $\lambda_m$4 and $\lambda_m$5” in 2D Poisson active learning	(Raissi et al., 2016)
PDE surrogates with high-dimensional outputs	Heat2 (Full) $\lambda_m$6, Poisson2 (Full) $\lambda_m$7, Fluid (Full) $\lambda_m$8 for D-MFDAL	(Wu et al., 2023)
Molecular discovery with GFlowNets	same mean Top-K energy as SF-GFN with about 20% of the budget on DNA aptamers; $\lambda_m$9 less budget than SF-GFN on AMP	(Hernandez-Garcia et al., 2023)
Rare-event reliability	computational savings of $f_H \equiv f_M$0, $f_H \equiv f_M$1, $f_H \equiv f_M$2, and $f_H \equiv f_M$3 versus single-fidelity EGRA	(Chaudhuri et al., 2019)
Airfoil optimization	cruise efficiency improvement $f_H \equiv f_M$4, take-off lift improvement $f_H \equiv f_M$5; only $f_H \equiv f_M$6 and $f_H \equiv f_M$7 of evaluated individuals require RANS	(Robledo et al., 17 Mar 2026)
Drug compound generation	“~50% improvement in mean binding free energy score”	(Eckmann et al., 2024)
Laser direct drive implosions	$f_H \equiv f_M$8 actively selected 2D simulations; burn-on $f_H \equiv f_M$9 for the 2D-optimised design	(Crilly et al., 28 Aug 2025)

Within PDE and operator-learning problems, the reported pattern is that low-fidelity data sharply improve sample efficiency when the cross-fidelity relation is informative. In the integro-differential example of operator-informed GP MFAL, $\sum_{(l,x)\in S} c_l \le B$0 low-fidelity and $\sum_{(l,x)\in S} c_l \le B$1 high-fidelity samples of the forcing yielded significantly improved posterior mean and tighter variance bands for the solution compared with the single-fidelity case using only $\sum_{(l,x)\in S} c_l \le B$2 high-fidelity samples (Raissi et al., 2016). In the 10D Poisson example, the same framework reported that ARD discovered effective dimensionality—dimensions $\sum_{(l,x)\in S} c_l \le B$3 and $\sum_{(l,x)\in S} c_l \le B$4 were active—even though low-fidelity data were active in all dimensions (Raissi et al., 2016). MRA-FNO extended the same broad pattern to operator learning, where relative $\sum_{(l,x)\in S} c_l \le B$5 error and NLL improved more rapidly under cost-aware multi-resolution querying than under random or single-resolution baselines, especially when cost annealing prevented early over-penalization of high-resolution queries (Li et al., 2023).

High-dimensional-output active learning shows similar gains. DMFAL reported $\sum_{(l,x)\in S} c_l \le B$6 speedups for topology optimization and $\sum_{(l,x)\in S} c_l \le B$7 speedups for CFD after training, relative to direct numerical solvers (Li et al., 2020). D-MFDAL reported lower nRMSE than MFHNP, NARGP, and DMFAL in Heat2, Poisson2, and Fluid under the Full setting, and its active-learning curves converged faster than DMFAL, BMFAL, and MF-BALD on Heat2/3, Poisson2/3, and Fluid tasks (Wu et al., 2023).

Rare-event simulation papers emphasize high-fidelity savings while retaining accuracy. mfEGRA located failure boundaries and estimated failure probabilities with the savings reported above relative to single-fidelity EGRA (Chaudhuri et al., 2019). The GP-corrected low-fidelity framework with dynamic subset-dependent U-functions reduced high-fidelity calls from $\sum_{(l,x)\in S} c_l \le B$8 to $\sum_{(l,x)\in S} c_l \le B$9 on the four-branch function, from $\sum_{t=1}^{T} c_{f_t}\le B$0 to $\sum_{t=1}^{T} c_{f_t}\le B$1 on the Rastrigin function, and from $\sum_{t=1}^{T} c_{f_t}\le B$2 to $\sum_{t=1}^{T} c_{f_t}\le B$3 on the Borehole function while matching the reported failure-probability estimates and coefficients of variation (Dhulipala et al., 2021). In noisy global design optimization, the stochastic radial basis function MFAL method reported aggregate-error improvements over high-fidelity-only surrogates on analytical benchmarks and positive design outcomes on the NACA hydrofoil, DTMB 5415 destroyer, and RoPax ferry problems (Pellegrini et al., 2022).

Discrete scientific discovery papers place more emphasis on diversity and cost efficiency than on pointwise surrogate accuracy. MF-GFN reported that multi-fidelity active learning with GFlowNets discovered high-scoring candidates at a fraction of the budget of its single-fidelity counterpart while maintaining diversity on molecular discovery tasks (Hernandez-Garcia et al., 2023). MF-LAL, which integrates a hierarchy of latent spaces with per-fidelity SVGP surrogates, reported mean ABFE values of $\sum_{t=1}^{T} c_{f_t}\le B$4 on BRD4(2) and $\sum_{t=1}^{T} c_{f_t}\le B$5 on c-MET after a seven-day active-learning budget, outperforming the stated single- and multi-fidelity baselines (Eckmann et al., 2024).

6. Assumptions, misconceptions, and open directions

MFAL inherits strong modeling assumptions from its surrogate class. In the operator-informed GP formulation, linearity of the operator $\sum_{t=1}^{T} c_{f_t}\le B$6 is essential for obtaining $\sum_{t=1}^{T} c_{f_t}\le B$7 and the associated cross-covariances (Raissi et al., 2016). In GP-based high-dimensional-output methods, Gaussian approximations and delta-method covariance calculations are central to tractability; BMFAL-BC explicitly notes that large nonlinearities or multimodality can bias covariance estimates and mutual information (Li et al., 2022). GP-based MFAL also encounters cubic scaling in the number of observations in dense settings, and several papers point to sparse approximations, variational sparse GPs, or recursive multi-fidelity schemes as remedies rather than presenting a fully resolved scalability solution (Raissi et al., 2016, Li et al., 2022).

A common misconception is that low fidelity is always helpful. The operator-informed GP paper states that if training yields $\sum_{t=1}^{T} c_{f_t}\le B$8, low-fidelity data are effectively ignored (Raissi et al., 2016). Another misconception is that MFAL must follow a strict hierarchical low$\sum_{t=1}^{T} c_{f_t}\le B$9high pipeline. MF-GFN explicitly states that it jointly samples $\Lambda$0, not in a strict hierarchical low$\Lambda$1high pipeline (Hernandez-Garcia et al., 2023). A further misconception is that diversity is incidental. In batch selection and discrete discovery, diversity is often a design requirement: BMFAL-BC uses batch mutual information to penalize highly correlated queries, and MF-GFN reports diversity metrics alongside mean Top-K scores (Li et al., 2022, Hernandez-Garcia et al., 2023).

Open directions are explicit across the papers. Reported extensions include more than two fidelity levels via recursive autoregression, multi-output GPs for systems of linear PDEs, alternative kernels including deep kernel learning, complex geometries and general linear boundary conditions, cost-aware active learning policies and information-theoretic acquisition functions, multi-fidelity multi-objective settings, hybrid discrete–continuous design spaces, continuous GFlowNets, and richer uncertainty aggregation (Raissi et al., 2016, Hernandez-Garcia et al., 2023). MRA-FNO adds automatic schedule selection for cost annealing as a future direction, while hierarchical trust-region MFAL for PDE-constrained optimization suggests richer ML models provided they remain paired with certification and efficient residual estimators (Li et al., 2023, Klein et al., 27 Mar 2025).

Taken together, these works depict MFAL not as a single algorithm but as a design pattern: a surrogate that links fidelities, an acquisition rule that quantifies value per cost, and an execution policy that respects budgets, parallel resources, or stopping rules. This suggests that the unifying technical question in MFAL is not merely how to interpolate across fidelities, but how to convert that cross-fidelity structure into the most informative next query for the highest-fidelity task.