---
title: 'Multi-Fidelity Surrogates: Techniques & Applications'
url: https://www.emergentmind.com/topics/multi-fidelity-surrogates-ba63cedf-77e6-4f7d-adaf-6331a368a75c
type: topic
---

# Multi-Fidelity Surrogates: Techniques & Applications

A multi-fidelity surrogate is a statistical or machine learning model that fuses datasets or simulation evaluations from several sources (“information sources” or “fidelity levels”) of varying cost and accuracy, with the goal of optimizing predictive accuracy within a constrained computational budget. The central paradigm is to leverage abundant but coarse low-fidelity (LF) data, correcting it with sparser but accurate high-fidelity (HF) data. This approach has become foundational in computational science and engineering for tasks such as simulation-based optimization, uncertainty quantification, rare-event estimation, and design of experiments. Theoretical, algorithmic, and empirical developments over the past decade have yielded a taxonomy of surrogate models capable of flexibly exploiting hierarchical, non-hierarchical, or heterogeneous fidelity arrangements.

## 1. Mathematical Frameworks for Multi-Fidelity Surrogates

Canonical multi-fidelity surrogates are built upon functional relationships between outputs at different fidelities. The simplest is the linear autoregressive model (“AR1” or recursive model), which expresses the HF quantity as
\[
f_{HF}(x) = \rho\,f_{LF}(x) + \delta(x)
\]
where $\rho$ is a scaling parameter and $\delta(x)$ is a (possibly nonlinear) discrepancy function [2404.11965]. This paradigm generalizes naturally to $K>2$ fidelities:
\[
f^{(k)}(x) = \rho_{k-1} f^{(k-1)}(x) + \delta_k(x), \quad k=1,\dots,K
\]
or, with nonlinear corrections, $f^{(k)}(x) = g_k(x, f^{(k-1)}(x)) + \delta_k(x)$. Discrepancy functions may be modeled as polynomials, Gaussian processes (GPs), neural networks, or other surrogates, and scale factors may be constant or input-dependent functions. Progressive, chain, or directed acyclic graph (DAG) topologies can be used to encode arbitrary conditional dependencies among fidelities [2510.13762, 2008.02672]. 

For domains with functional or field outputs (e.g., time series or spatial PDE solutions), outputs are often projected onto low-dimensional linear (PCA/POD) or nonlinear manifolds, and multi-fidelity surrogates are formulated on the manifold coordinates [2408.17075, 2109.11374]. With heterogeneous input spaces (e.g., where LF and HF solvers differ in parameterization), explicit maps are learned between input domains prior to output fusion [2403.13136]. For uncertainty-aware surrogates, aleatoric and epistemic uncertainty are jointly modeled via Bayesian or bootstrap ensemble constructs, providing rigorous predictive intervals [2401.06447, 2312.02575].

## 2. Algorithmic and Architectural Variants

### 2.1 Gaussian Process and Kernel Methods

The AR1 co-kriging model—originating with Kennedy & O’Hagan (2001)—remains the standard in GP-based surrogates, where the cross-correlation between fidelities yields joint GP priors with block-structured kernels. For $L$ levels:
\[
\begin{cases}
Y_1(x) \sim \mathcal{GP}(0, k_1(x,x'))\\
Y_k(x) = \rho_{k-1} Y_{k-1}(x) + \delta_k(x),\ \delta_k \sim \mathcal{GP}(0, k_{\delta_k})(x,x')
\end{cases}
\]
and predictions/marginals can be computed block-wise [2404.11965]. Nonlinear AR (NARGP) and delay-augmented kernels enable flexible modeling of nonlinear, phase-shifted, or derivative relationships [2404.11965]. Recent models incorporate spatially-varying trust weights for heteroscedastic fusion, e.g., the “MAST” approach, which spatially blends corrected LF and HF predictions via distance-based weights and explicitly propagates variance [2602.20974].

Support Vector Regression (Co_SVR) adopts an SVR model for the discrepancy, using tuned block kernels to draw on both LF and HF sample structures [1906.09439]. Fully linear least-squares surrogates (LS-MFS) can also be constructed by embedding LF predictions as regression bases alongside polynomial discrepancies [1705.02956].

### 2.2 Deep Learning Approaches

Multi-fidelity surrogates based on neural architectures provide scalability for high-dimensional and functional data. Progressive residual correction NNs use a stack of encoder-decoder pairs for each fidelity level, where encoders handle arbitrary input modalities and decoders perform regression to cumulative residuals; each level is trained sequentially with weights frozen below to prevent knowledge degradation [2510.13762]. Neural process models with decoder-level aggregation offer explicit residual learning, coupling inference between decoded low- and high-fidelity levels for accurate transfer to out-of-distribution regions [2402.18846]. LSTM-based surrogates, both in hierarchical and parallel layout, efficiently model time-dependent and parametric systems, with correction or fusion occurring at the level of sequence latent states or outputs [2208.03115, 2309.00325]. CNNs with transfer learning (multi-fidelity up-projection) exploit parameter-efficient adaptation for field outputs, guided by optimal sample allocation under multilevel Monte Carlo theory [2105.00856].

For uncertainty-aware surrogates, Bayesian neural networks (GPBNN) combine a GP emulator for the LF level with a BNN for the HF correction, propagating and integrating model uncertainty via Gauss-Hermite quadrature and HMC inference [2312.02575].

### 2.3 Surrogates with Domain/Mesh/Output Heterogeneity

Heterogeneous input spaces are reconciled by learning linear (or nonlinear) maps between higher- and lower-dimensional input spaces, allowing joint AR1 co-kriging on a common domain [2403.13136]. For functional outputs differing in mesh resolution or structure, methods employ dimensionality reduction to encode outputs to latent spaces, with fusion, corrective, or mapping surrogates operating therein [2408.17075]. Tensor kernel GP surrogates and co-kriging for basis coefficients extend this to time-series and field outputs [2109.11374].

## 3. Training, Adaptive Sampling, and Active Learning

Multi-fidelity surrogate training typically uses sequential or all-at-once (for DAG/fusion schemes) optimization. GP-based models maximize the log-marginal likelihood, possibly via block or sparse methods to handle large sample sets [2404.11965]. Neural architectures rely on stochastic gradient optimization (Adam, Adamax), regularization, and cross-validation for hyperparameter tuning [2510.13762, 2208.03115]. For deep Bayesian models, posterior sampling (e.g., HMC, NUTS), or variational inference is used to quantify and calibrate predictive uncertainty [2312.02575].

Practical active learning algorithms are often embedded in optimization or UQ loops to sequentially select the most informative new data points at an optimal fidelity, using uncertainty-based acquisition (lower confidence bound) or information gain metrics, and benefit-cost trade-offs that account for simulation cost differentials [2212.03375, 2202.06902]. Adaptive sample allocation guided by multilevel Monte Carlo variance/cost minimization ensures optimal split between LF and HF evaluations [2105.00856].

## 4. Comparative Performance and Benchmarking

Across synthetic functions, PDE benchmarks, and industrial applications (e.g., air-pollution, melt pool geometry, aerodynamic shape, climate emulation), multi-fidelity surrogates consistently outperform single-fidelity approaches for a fixed computational budget [2510.13762, 2408.17075, 2404.11965, 2403.13136]. Progressive neural surrogates reduce relative error by up to an order of magnitude when integrating additional (multi-modal) data streams [2510.13762]. In rare-event estimation, multi-fidelity surrogates within subset simulation can reduce the number of HF simulations by two orders of magnitude compared to HF-only approaches, with negligible loss in final accuracy [2212.03375]. Uncertainty-aware and ensemble approaches provide well-calibrated prediction intervals with empirically controlled coverage [2401.06447, 2312.02575]. In the data-scarce regime, all-at-once DAG surrogates and neural residual methods demonstrate order-of-magnitude gains over hierarchical discrepancy chains by efficiently aggregating information from non-nested and non-peered sources [2008.02672, 2402.18846].

## 5. Advantages, Limitations, and Open Challenges

Advantages of multi-fidelity surrogates include: (i) dramatic reduction in the number of expensive HF evaluations for a fixed error; (ii) applicability to arbitrary or heterogeneous data modalities; (iii) online flexibility—robust prediction is feasible given only a subset of available data streams; (iv) explicit or ensemble-based uncertainty quantification; (v) mitigation of catastrophic forgetting via residual or progressive training [2510.13762, 2602.20974].

Key limitations and challenges are as follows: (i) hyperparameter tuning for complex surrogates remains nontrivial and often problem-dependent; (ii) synchronization and alignment of heterogeneous datasets are required; (iii) interpretability is limited for deep/black-box surrogates; (iv) offline training (especially for deep ensembles) can be expensive; (v) current methods may degrade if LF/HF correlation is weak or nonlinearities are extreme. For high-dimensional outputs, tensorized covariance and manifold alignment remain active areas of research [2408.17075].

Extensions under active investigation include: integration of fully Bayesian decoding for calibration-free uncertainty quantification, handling arbitrary patterns of missing modalities through attention-like mechanisms, embedding physics-informed constraints (PINNs) for physical consistency, and coupling surrogates to inverse or design optimization loops [2510.13762, 2402.18846].

## 6. Illustrative Algorithms and Architectures

### 6.1 Progressive Multi-Fidelity Surrogate—Block Diagram

Imagine a horizontal stack of $K+1$ blocks, each with an encoder for its input modality. All encoders output latent codes to a vertical bus so that the $k^\text{th}$ block receives all codes up to level $k$; the decoder at each level outputs a residual, which is additively summed with previous outputs for the final prediction. Earlier encoder/decoder weights are frozen as levels progress, mitigating knowledge degradation [2510.13762].

### 6.2 Training Pseudocode for Progressive Surrogates

```python
for k in range(K+1):
    freeze_previous_layers(k)
    initialize_current_layer(k)
    for epoch in range(max_epochs):
        for batch in data[k]:
            # Encode all fidelities
            Z = [Phi[l](x[l]) for l in range(k+1)]
            # Decode residual
            delta = Psi[k](concat(Z))
            # Add to previous surrogate
            y_pred = f_prev(x[:k]) + delta
            # Compute loss + L2 regularization
            loss = MSE(y_pred, y_true) + lambda_Phi * ||W_Phi[k]||^2 + lambda_Psi * ||W_Psi[k]||^2
            # Backpropagate and update current W_Phi, W_Psi
    update_surrogate(k)
# Final surrogate is f^{(K)}
```
[2510.13762]

### 6.3 All-at-Once Multifidelity DAG Optimization

The loss is a nonlinear least squares term over all sources, each node’s output a function of its parents and a local bias. Forward and backward sweeps efficiently propagate feature evaluations and aggregated residuals through the graph for Gauss–Newton or gradient-based optimization [2008.02672].

## 7. Representative Benchmark Results

| Problem             | Progressive MF (Rel. Err) | All-at-once DAG MF (Rel. Err) | Baseline HF Only |
|---------------------|--------------------------|-------------------------------|------------------|
| Reaction–Diffusion  | 18.6%                    | $\sim$1e-2 (DAG, 3 pts)       | 81.3% (params)   |
| Navier–Stokes       | 6.50%                    | —                             | 12.3% (params)   |
| Air Pollution       | 13%                      | —                             | 72% (temp)       |

Empirical studies repeatedly show multi-fidelity surrogates yielding order-of-magnitude error reductions and extreme computational savings, particularly when the correlation between LF and HF is moderately strong and the LF domain covers the input space of interest [2510.13762, 2401.06447, 2212.03375, 2008.02672].

---

**References:**  
- "Progressive multi-fidelity learning for physical system predictions" [2510.13762]  
- "MFNets: Data efficient all-at-once learning of multifidelity surrogates as directed networks of information sources" [2008.02672]  
- "A survey on multi-fidelity surrogates for simulators with functional outputs: unified framework and benchmark" [2408.17075]  
- "Multi-Fidelity Residual Neural Processes for Scalable Surrogate Modeling" [2402.18846]  
- "MAST: A Multi-fidelity Augmented Surrogate model via Spatial Trust-weighting" [2602.20974]  
- "Worst-Case Learning under a Multi-fidelity Model" [2406.14418]  
- "Multi-fidelity surrogate with heterogeneous input spaces for modeling melt pools in laser-directed energy deposition" [2403.13136]  
- "Uncertainty-aware multi-fidelity surrogate modeling with noisy data" [2401.06447]  
- "General multi-fidelity surrogate models: Framework and active learning strategies for efficient rare event simulation" [2212.03375]  
- "Multi-fidelity surrogate modeling using long short-term memory networks" [2208.03115]  
- "Multi-fidelity reduced-order surrogate modeling" [2309.00325]  
- "A support vector regression-based multi-fidelity surrogate model" [1906.09439]  
- "Multi-fidelity surrogate modeling for time-series outputs" [2109.11374]  
- "Transfer Learning on Multi-Fidelity Data" [2105.00856]  
- "Multi-Fidelity Surrogate Based on Single Linear Regression" [1705.02956]

Source: https://www.emergentmind.com/topics/multi-fidelity-surrogates-ba63cedf-77e6-4f7d-adaf-6331a368a75c