Multi-Fidelity Transfer Learning Methods

• Multi-fidelity transfer learning is a set of methods that use abundant, low-cost, low-fidelity data to enhance scarce high-fidelity predictions by transferring model parameters, outputs, or residual corrections.
• It employs mechanisms such as sequential pretraining, joint fidelity-conditioned learning, and probabilistic operator models across disciplines like CFD, materials science, and cosmology.
• The approach improves efficiency and accuracy while reducing costs, yet it faces challenges like misaligned fidelity correlations and the need for robust uncertainty quantification.

Multi-fidelity transfer learning denotes a class of methods that use abundant, cheap, and imperfect low-fidelity information to improve learning on scarce, expensive, and accurate high-fidelity targets. Across recent work, “low fidelity” may mean coarser meshes, larger time steps, looser tolerances, reduced physical detail, lower-order discretizations, cheaper DFT functionals, smaller simulation volumes, or simpler wake and flow models, while “high fidelity” denotes the more accurate but more costly counterpart (Deng et al., 2023). The transfer mechanism is not uniform: some methods transfer model parameters, some transfer low-fidelity outputs or residuals, some learn explicit cross-fidelity operators, some condition a shared model on fidelity labels, and some perform online statewise transfer during optimization or reinforcement learning (Gong et al., 2021, Agrawal et al., 23 Mar 2025).

1. Conceptual foundations and problem structure

A standard problem formulation involves two or more datasets,

$$\mathcal D^i=\{x_n^i,y_n^i\}_{n=1}^{N^i}, \qquad i=1,\dots,\tau,$$

with higher fidelities usually more accurate and more expensive, so that $N^1 > N^2 > \cdots > N^\tau$, and the objective is to estimate the highest-fidelity map $f^\tau$ from limited high-fidelity observations while exploiting cheaper lower-fidelity information (Deng et al., 2023). In practice, the datasets may be nested, non-nested, paired, partially paired, or entirely unpaired, and outputs may be aligned, unaligned, low-dimensional, or high-dimensional tensor fields (Deng et al., 2023, Ibrahim et al., 2024, Saoulis et al., 27 May 2025).

The literature also draws an important distinction between transfer learning and multi-fidelity learning. In the materials-calibration setting, transfer learning is defined as pretraining a neural network on a large low-fidelity dataset and then transferring the learned weights to initialize training on the small high-fidelity dataset, whereas multi-fidelity learning incorporates low-fidelity information directly into the supervised task on the high-fidelity data, through the input features, the target labels, or both (Gong et al., 2021). That distinction is useful but not universal. Other work collapses the two notions by treating multi-fidelity learning itself as a transfer problem over model parameters, latent representations, or online experience streams (Bhola et al., 2022, Agrawal et al., 23 Mar 2025).

A recurring structural assumption is cross-fidelity correlation. Some papers encode this through explicit discrepancy models, as in

$$\Delta H_f^{\mathrm{diff}}=\Delta H_f^{\mathrm{exp}}-\Delta H_f^{\mathrm{DFT}},$$

followed by reconstruction of the calibrated target from low-fidelity prediction plus learned correction (Gong et al., 2021). Others encode it through autoregression,

$f^h(x)=\rho f^l(x)+f^r(x),$

or its tensor generalization,

$Y^h(x)=Y^l(x)\times W + Y^r(x),$

which treats high fidelity as a transformed low-fidelity field plus residual discrepancy (Deng et al., 2023). In reinforcement learning, correlation is replaced by local policy agreement, so that transfer occurs only when a low-fidelity policy is aligned with the evolving high-fidelity policy (Agrawal et al., 23 Mar 2025).

2. Principal transfer mechanisms

A first family is sequential pretraining and fine-tuning. In controlled transfer learning for airfoil shape optimization, the policy network is trained first in a low-fidelity potential-flow environment and then transferred to a high-fidelity steady RANS CFD environment, with switching triggered by a reward-variance ratio $\beta_e$ and cutoff $\Gamma=0.3$, rather than by a fixed episode count (Bhola et al., 2022). In scientific image emulation, a StyleGAN2 trained on 120,000 small-scale 21 cm images is converted into a large-scale generator and then adapted using only 320 large-scale target images, with additional mechanisms such as cross-domain correspondence and frozen discriminator layers to preserve small-scale realism and diversity (Diao et al., 6 Feb 2025). In PDE surrogates, a CNN or recurrent residual U-Net is pretrained on low-fidelity coarse-grid outputs, then a high-fidelity output layer is trained, and finally the whole model is fine-tuned on a small high-fidelity set (Song et al., 2021, Jiang et al., 2022).

A second family is joint fidelity-conditioned learning. In optical-spectrum prediction, a MEGNet-style graph neural network is trained jointly on OPT and MBJ DFT data by encoding the fidelity level as an integer and passing it through a trainable embedding layer used as a global state feature; this fidelity embedding outperforms frozen-encoder transfer because the entire graph representation can become fidelity-aware (Ibrahim et al., 2024). For interatomic potentials, SevenNet-MF introduces fidelity-conditioned linear layers inside an equivariant graph neural network, together with fidelity-specific energy shift and scale values, so that a shared network learns a common PES trend while fidelity-dependent parameters model deviations for PBE, meta-GGA, or higher-fidelity labels (Kim et al., 2024).

A third family is residual and discrepancy learning. In formation-enthalpy calibration, the best model is a random forest trained on $\Delta H_f^{\mathrm{diff}}$ with composition descriptors and $\Delta H_f^{\mathrm{DFT}}$ as an additional feature, producing

$N^1 > N^2 > \cdots > N^\tau$0

The paper explicitly contrasts this with weight-transfer neural baselines and argues that, when low-fidelity values are available at inference time, passing information through data values can be more effective than passing it through pretrained parameters (Gong et al., 2021). A related generalization is the deterministic/Bayesian three-model decomposition

$N^1 > N^2 > \cdots > N^\tau$1

in which a deterministic low-fidelity model learns the broad trend, a transfer map $N^1 > N^2 > \cdots > N^\tau$2 adapts it to high fidelity, and a Bayesian residual model accounts for remaining discrepancy and uncertainty (Yi et al., 2024).

A fourth family is probabilistic cross-fidelity operator learning. GAR and CIGAR replace the scalar transfer coefficient of classical autoregression with a Tucker-structured linear map between low- and high-fidelity tensor outputs, combine that with tensor-variate Gaussian-process priors, and extend the formulation to non-subset input designs by introducing latent low-fidelity values and a closed-form integrated likelihood (Deng et al., 2023). This is not pretrain/fine-tune transfer, but direct probabilistic transfer in output space.

3. Adaptive transfer in optimization, reinforcement learning, and hyperparameter search

In multi-fidelity reinforcement learning, recent work has challenged the assumption that low-fidelity models can be arranged in a reliable hierarchy. The adaptive framework of (Agrawal et al., 23 Mar 2025) treats each fidelity as its own PPO learner and transfers low-fidelity trajectories to the high-fidelity learner only when the corresponding low-fidelity policy is locally aligned with the high-fidelity policy. Alignment is computed by cosine similarity between the mean action distributions,

$N^1 > N^2 > \cdots > N^\tau$3

with a threshold that tightens over training from an angle of $N^1 > N^2 > \cdots > N^\tau$4 toward $N^1 > N^2 > \cdots > N^\tau$5. Model selection is then performed by an $N^1 > N^2 > \cdots > N^\tau$6-greedy rule over $N^1 > N^2 > \cdots > N^\tau$7, and only contiguous aligned low-fidelity subsequences are appended to the high-fidelity buffer (Agrawal et al., 23 Mar 2025). The closely related ALPHA framework makes the same non-hierarchical point explicitly: heterogeneous low-fidelity models may each be accurate in different regions of the design space, so transfer should be governed by local policy agreement rather than a fixed LF$N^1 > N^2 > \cdots > N^\tau$8HF schedule (Agrawal et al., 2024).

This online, statewise transfer differs sharply from earlier sequential switching strategies. In the airfoil CTL framework, the transfer object is the policy network parameters, and the main innovation is deciding when to switch from low to high fidelity via a convergence statistic on recent rewards (Bhola et al., 2022). In ALPHA and the 2025 adaptive octocopter framework, the transfer object is not merely an initialization but a filtered stream of low-fidelity behavior and experience reused throughout training (Agrawal et al., 2024, Agrawal et al., 23 Mar 2025).

A related but distinct development appears in cost-sensitive multi-fidelity Bayesian optimization. CMBO models partial learning curves over epochs, uses a transfer-learned probabilistic extrapolator trained on historical learning-curve datasets, and chooses configurations by maximizing a utility-aware acquisition,

$N^1 > N^2 > \cdots > N^\tau$9

where a user-defined utility such as

$f^\tau$0

trades off incumbent BO performance against BO cost (Lee et al., 2024). This makes transfer learning a component of multi-fidelity BO at the surrogate level: a pretrained model of learning-curve evolution is used to decide both whether a configuration should be continued and whether BO itself should be stopped.

4. Surrogate modeling, operator learning, and uncertainty-aware fusion

Scientific surrogate modeling supplies many of the clearest formulations of multi-fidelity transfer. In porous-media flow, low- and high-fidelity simulations correspond to coarse and fine meshes, and transfer learning is implemented by training a CNN on coarse outputs, swapping in a fine-resolution output layer, and then fine-tuning on fine-grid data (Song et al., 2021). In 3D two-phase subsurface flow, the recurrent residual U-Net uses the high-fidelity geomodel as input at all stages, pretrains on 2,500 low-fidelity coarse simulations, retrains only the output layer on 200 high-fidelity simulations, and then fine-tunes the full network, thereby keeping the spatial representation fixed while transferring dynamics and decoder structure across fidelities (Jiang et al., 2022). In Fourier Neural Operator transfer, the same FNO backbone is pretrained on abundant low-fidelity fields and then fine-tuned on scarce high-fidelity fields, exploiting resolution invariance to avoid explicit coarse-to-fine bridge networks (Lyu et al., 2023).

Physics-informed transfer constitutes another distinct variant. MF-PIDNN uses an approximate governing equation as the low-fidelity source, rather than low-fidelity labels. A physics-informed DNN is first trained on the approximate residual,

$f^\tau$1

then only the last one or two layers are fine-tuned on sparse high-fidelity data through a supervised MSE term (Xu et al., 2020). This makes low-fidelity physics itself the transferable prior.

Uncertainty-aware variants fall into several categories. MF-BayNet performs staged transfer across XFoil, coarse-mesh RANS, and fine-mesh RANS, but retains Bayesian weight distributions and Monte Carlo prediction at every stage. Its predictive distribution is written as

$f^\tau$2

and transfer carries posterior parameter means and variances forward while freezing selected layers (Vaiuso et al., 2024). GAR/CIGAR deliver closed-form Gaussian predictive distributions for tensor outputs and explicitly support non-aligned outputs and non-subset designs (Deng et al., 2023). The aerodynamic autoencoder-transfer framework instead freezes a low-fidelity encoder, fine-tunes the decoder plus an adaptation layer, and calibrates vector-valued predictive bands with Multi-Split Conformal Prediction, using normalized residual scores and median aggregation across repeated train/calibration splits (Nieto-Centenero et al., 15 Dec 2025). The practical deterministic/Bayesian fusion framework likewise reserves uncertainty quantification for the high-fidelity residual, rather than for the full low-fidelity-to-high-fidelity map (Yi et al., 2024).

5. Representative application domains and empirical record

The empirical record spans materials, atomistics, fluid mechanics, design optimization, and cosmology. In materials-property calibration, the multi-fidelity random forest for experimental formation enthalpy achieves a held-out MAE of 0.0617 eV/atom, compared with 0.0955 eV/atom for the Materials Project corrected-PBE baseline, and on a 122-material subset it yields 0.0542 eV/atom, outperforming PBE, PBEsol, and SCAN baselines listed in the study (Gong et al., 2021). In optical-spectrum prediction, joint fidelity-conditioned learning reduces the median MAE of the unnormalized MBJ spectrum by 24.6% for $f^\tau$3 and 25.0% for $f^\tau$4 relative to the single-fidelity model, slightly outperforming frozen-encoder transfer (Ibrahim et al., 2024). For high-accuracy interatomic potentials, simultaneous multi-fidelity training on GGA and meta-GGA data is reported to be more effective than transfer learning or $f^\tau$5-learning, and the same framework is stated to extend up to coupled-cluster targets (Kim et al., 2024).

Engineering design and aerodynamic modeling show similarly diverse transfer modes. Controlled low-to-high fidelity RL for airfoil shape optimization reduces the number of expensive high-fidelity evaluations by over 30% while preserving the reported final drag coefficients (Bhola et al., 2022). WakeNet reaches 99.8% accuracy on Curl wakes using only 100 Curl wakes after Gaussian-model pretraining, and in 15-turbine layout optimization it reproduces a 60.83% power gain versus 61.28% for FLORIS while reducing runtime from 2160 min to 13 min (Anagnostopoulos et al., 2023). In transonic aerodynamic loads, MF-BayNet reduces total prediction error to 4.91%, compared with 7.61% for Co-Kriging and larger errors for single-fidelity BNNs (Vaiuso et al., 2024). In 3D subsurface flow, the multifidelity recurrent R-U-Net uses 2500 low-fidelity runs and 200 high-fidelity runs, for about a 90% reduction in training simulation cost relative to a 2500-high-fidelity reference surrogate, while remaining much more accurate than the low-fidelity simulator itself (Jiang et al., 2022).

Cosmology provides both generative and inference-oriented examples. For survey-scale 21 cm lightcone generation, a GAN pretrained on 30,000 cheap small-scale simulations and adapted with only 80 large-scale simulations yields percent-level accuracy on many small-scale statistics and costs about 1.14\times 10^4 CPU core hours, compared with estimated all-large-scale alternatives in the range 15\times 10^4 to 90\times 10^4 core hours, summarized as a 90%–99% cost saving (Diao et al., 6 Feb 2025). For simulation-based inference on CAMELS, pretraining on dark-matter-only simulations reduces the required number of high-fidelity hydrodynamical simulations by a factor between 8 and 15, depending on posterior dimensionality and evaluation metric (Saoulis et al., 27 May 2025).

6. Limitations, misconceptions, and current directions

A persistent misconception is that multi-fidelity transfer learning is identical to low-fidelity pretraining followed by high-fidelity fine-tuning. The literature sampled here is broader. Some methods transfer through data values rather than parameters (Gong et al., 2021), some encode an explicit probabilistic cross-fidelity operator (Deng et al., 2023), some use fidelity embeddings inside a jointly trained model (Ibrahim et al., 2024, Kim et al., 2024), and some perform online, local, state-conditional transfer rather than one-time initialization (Agrawal et al., 23 Mar 2025).

A second misconception is that low-fidelity information is automatically helpful. Several papers report clear failure modes. In non-hierarchical RL, low-fidelity models with heterogeneous error distributions can inject inconsistent policy-update signals and degrade convergence under rigid schedules (Agrawal et al., 2024, Agrawal et al., 23 Mar 2025). In materials calibration, the multi-fidelity scheme assumes that $f^\tau$6 is available at inference time; without low-fidelity values for deployment candidates, that formulation cannot be used directly (Gong et al., 2021). In cosmological SBI, zero-shot transfer from $f^\tau$7-body to hydrodynamical simulations performs extremely poorly, so target-task fine-tuning remains essential (Saoulis et al., 27 May 2025). In few-shot GAN adaptation, the largest spatial modes remain the hardest to learn, and mode-collapse risk is only mitigated, not eliminated (Diao et al., 6 Feb 2025). This suggests that fidelity correlation is necessary but not sufficient; the form of transfer must also match the mismatch structure.

Uncertainty quantification remains uneven. GP-based and conformal approaches provide explicit predictive bands, but even there the strongest guarantees may weaken in practical modifications such as median-aggregated multi-split conformal calibration (Nieto-Centenero et al., 15 Dec 2025). Bayesian neural variants provide predictive uncertainty, yet the decomposition into epistemic and aleatory components is often incomplete or only conceptual (Vaiuso et al., 2024). This suggests that uncertainty-aware multi-fidelity transfer is advancing, but not yet standardized.

Several open directions are explicit in the cited work. GAR notes the absence of active learning and the scalability challenge of non-subset training (Deng et al., 2023). ALPHA proposes alternative alignment metrics and cost-weighted model-choice probabilities when multiple low-fidelity models are aligned (Agrawal et al., 2024). CMBO leaves the utility function user-specified rather than learned from interaction (Lee et al., 2024). The aerodynamic autoencoder framework points toward adaptive or online transfer learning in which the latent space itself can change, rather than remaining frozen after low-fidelity pretraining (Nieto-Centenero et al., 15 Dec 2025). A plausible implication is that the field is moving from static fidelity hierarchies toward adaptive, uncertainty-aware, and structurally heterogeneous transfer mechanisms, with the strongest gains appearing when low- and high-fidelity sources share substantial latent structure but differ in cost, coverage, or local validity.