Multifidelity & Multitask Modeling

• Multifidelity and multitask modeling are methodologies that integrate high- and low-fidelity data along with related tasks to construct accurate, cost-efficient surrogates.
• They employ both model-free and model-based strategies, utilizing techniques like autoregressive structures and Gaussian processes to optimize prediction accuracy.
• Advanced data fusion, Bayesian uncertainty quantification, and adaptive sampling techniques are key to mitigating negative transfer and enhancing robust predictive performance.

Multifidelity and multitask modeling are key methodologies for constructing predictive models by leveraging heterogeneous data sources and exploiting relatedness among different prediction problems. These frameworks are critical when expensive high-fidelity data are scarce but inexpensive low-fidelity or auxiliary data are abundant, and when multiple related prediction tasks (e.g., output properties, control variables, scenarios) can inform each other’s inference. The central objective is to improve prediction accuracy, efficiency, or uncertainty quantification beyond what is possible with separate, task-specific, or fidelity-specific models.

1. Mathematical and Conceptual Foundations

Multifidelity modeling integrates data from simulation or experiment conducted at different levels of fidelity (e.g., low-fidelity $\ell$, high-fidelity $h$), where fidelity refers to the granularity, complexity, or realism of a data source. The typical goal is to construct an accurate surrogate for the high-fidelity target $f^h(x)$ using small $n_h$ sets of $(x, f^h(x))$ and large $n_\ell$ $(x, f^\ell(x))$. Multitask modeling, by contrast, addresses a collection of $T$ related tasks, predicting $Y_t$ from $X_t$ for each $t=1, \dots, T$. Multifidelity and multitask modeling often overlap: one can think of each fidelity or task as an “output head” in a composite model, and methods for sharing information across outputs are central to both paradigms.

A canonical multifidelity structure is the autoregressive model:

$f^h(x) = \rho\, f^\ell(x) + \delta(x)$

where $\rho$ is a scaling parameter and $\delta(x)$ a bias (possibly nonlinear or input-varying). In multitask Gaussian processes (GPs), the regression functions are coupled via shared or task-specific kernels and cross-covariances:

$\mathbb{E}[f_t(x)f_s(x')] = K_{ts}(x,x')$

with flexible parameterizations to encode, e.g., covariance through latent factors or explicit task clusters.

Both fields are unified by their treatment of the response space as a set of related outputs, for which joint statistical modeling and cross-task/fidelity transfer are advantageous. In many applications, fidelity and task structure are both present and must be modeled together.

2. Model-Free and Model-Based Approaches

Two broad classes dominate: model-free (statistical) and model-based (mechanistic/decision-theoretic).

• Model-free approaches rely on regression or machine learning techniques without built-in assumptions about the data-generating mechanism. For example, (Schlicht et al., 2012, Schlicht et al., 2014) use locally weighted (LW) regression on high-fidelity data:

$$\hat{A}_i^\text{te} = \sum_{j=1}^{N_h^\text{tr}} z_{i,j} A_{h,j}^\text{tr}$$

with weights $z_{i,j}$ based on Euclidean distance in the state/feature space. Multifidelity extensions add predictions from a low-fidelity regression as features or covariates:

$$\hat{A}^{\text{te}} = R'_h(S^{\text{te}}, R_\ell(S^{\text{te}}))$$

This general strategy is prevalent across tree ensembles with soft sharing (Ibrahim et al., 2022), neural networks with selector inputs (Appleton et al., 21 Aug 2024), or kernel methods with explicit task indicators (Chen et al., 2023).

• Model-based approaches introduce domain knowledge—such as human decision rules or physical laws. For human-in-the-loop pilot modeling (Schlicht et al., 2012), pilots’ decisions are described via game-theoretic constructs (level-$k$ reasoning) and bounded rationality, with utility weights reflecting trade-offs (e.g., heading deviation vs. collision risk). Fusion then proceeds by estimating these parameters jointly from both high- and low-fidelity data using likelihood-based methods (e.g., MAP estimation):

$$w^*_h = \arg\max_{w_h} \left[ \ln p(w_h) + \sum_n \ln p(A^n_h|w_h) \right]$$

Bayesian approaches further integrate out uncertainty in weights by marginalizing over high- and low-fidelity posterior likelihoods.

Both frameworks can be equipped for multitask settings, either by stacking multiple regression heads (deep multitask learning), introducing latent task embeddings (nonparametric Bayesian mixture and subspace models) (Passos et al., 2012), or treating each task/fidelity as a draw from a flexible prior (Fisher et al., 31 Jan 2024).

3. Data Fusion and Adaptive Sampling

Central to multifidelity and multitask modeling is data fusion: the judicious combination of disparate data. Strategies include:

• Augmenting feature spaces: Using low-fidelity predictions as input or covariate for high-fidelity regressors (as in multifidelity LW regression or tree ensemble regularization (Schlicht et al., 2012, Ibrahim et al., 2022)).
• Joint likelihood estimation: For example, in multitask GP regression, the covariance structure is designed to leverage similarities without imposing explicit dominance or hierarchy among tasks (Fisher et al., 31 Jan 2024, Chen et al., 2023, Wu et al., 2022).
• Latent variable embeddings: LVGP-based methods learn a non-hierarchical latent space jointly for all fidelities, projecting the fidelity indicator to a location that governs correlation and information sharing (Chen et al., 2023).
• Active learning and subset simulation: Some frameworks adaptively select where and which fidelity to sample next, based on pre-posterior analysis of improvement per unit cost, or by updating acquisition functions and uncertainty surfaces (Chen et al., 2023, Dhulipala et al., 2021, Dhulipala et al., 2022).

This fusion is critical in scenarios where task or fidelity correspondence is imperfect or where fidelity/task “assignment” is a latent variable rather than a fixed deterministic rule.

4. Uncertainty Quantification, Bayesian Formulations, and Expressivity Trade-offs

Advanced models increasingly embed uncertainty quantification directly in their outputs and adopt Bayesian machinery for inference:

• Deep Gaussian Processes (DGPs) offer hierarchical composition and propagate uncertainty through each fidelity layer, allowing both linear and highly nonlinear correlations between fidelities to be learned (Cutajar et al., 2019, Hebbal et al., 2020). The evidence lower bound (ELBO) becomes the central objective, regularized by KL terms from prior/posterior divergence.
• Bayesian multitask models and hierarchical neural processes treat latent dependencies (e.g., latent task/fidelity embeddings) as random variables with variational posteriors, promoting scalability and proper handling of non-nested, high-dimensional data (Wu et al., 2022).
• Expressivity trade-offs: As articulated in (Yi et al., 21 Jul 2024), there exists a balance between the complexity of transfer learning (how much of $f^h(x)$ is captured as a function of $f^\ell(x)$) and the residual’s modeling burden. A simpler linear transfer suffices if the cross-fidelity dependence is well-captured, otherwise a more complex (Bayesian) model is necessary for the residual.

Explicit uncertainty quantification is essential in UQ-heavy domains such as rare event simulation (Dhulipala et al., 2021), reliability assessment (Dhulipala et al., 2022), and scientific modeling with propagation of distributional uncertainty ((Giannoukou et al., 14 Jul 2025), which fuses full conditional response distributions).

5. Practical Applications and Empirical Findings

Multifidelity and multitask modeling underpin a diverse set of applications:

• Human-in-the-loop simulation: In pilot interaction modeling, fusing online low-fidelity and limited high-fidelity data gives high-accuracy prediction of interacting decisions under data constraints (Schlicht et al., 2012, Schlicht et al., 2014). Bayesian model-based multifidelity outperforms model-free and high-fidelity–only methods especially when parameter similarity across fidelities holds.
• Computational science, engineering design, and UQ: DGPs and Bayesian models are leveraged to integrate coarse and fine simulations for PDEs, reliability analysis, power grid security games, and climate modeling (Hebbal et al., 2020, Penwarden et al., 2021, Wu et al., 2022). Active learning/fidelity selection strategies minimize costly simulation calls in rare event estimation or structural reliability (Dhulipala et al., 2021, Dhulipala et al., 2022).
• Materials science and chemistry: Multitask GP regression enables CCSD(T)-level property prediction at a fraction of the quantum chemical computation cost by harnessing abundant, structurally diverse DFT data, even with fully heterogeneous (nonaligned) secondary data (Fisher et al., 31 Jan 2024).
• Multimodal multi-property prediction: In material design for energetic materials, multi-task neural networks (“MT-NN”) leverage both computational and experimental properties, using selectors to specialize the output while sharing layer representations, yielding particular gains for data-scarce properties (Appleton et al., 21 Aug 2024).
• Stochastic simulation: MF-GLaM extends multifidelity emulation of deterministic models to the full conditional response distribution of stochastic simulators, estimating both LF and correction (discrepancy) parameters with polynomial chaos expansions and maximum likelihood, achieving similar response accuracy with significantly reduced HF sampling cost (Giannoukou et al., 14 Jul 2025).

Empirical evaluations across domains consistently show that fusing low- and high-fidelity data yields either improved predictive accuracy for a set computational budget, or comparable accuracy at lower cost, especially when the underlying cross-fidelity or cross-task correlation is strong.

6. Challenges, Limitations, and Open Problems

Despite their advances, multifidelity and multitask modeling face several persistent challenges:

• Fidelity/task alignment: Discrepancies in input domains, parameterization, or unmodeled physics necessitate sophisticated input-mapping modules (e.g., calibrated via multi-output GPs (Hebbal et al., 2020)) or latent embeddings (Chen et al., 2023). Misaligned or sparse data can degrade fusion effectiveness.
• Model/hyperparameter complexity: Nonparametric and hierarchical models (e.g., Dirichlet Process mixtures, Infinite Factor models (Passos et al., 2012), deep GPs (Hebbal et al., 2020)) require careful regularization and variational approximations to avoid overfitting and numerical instability.
• Uncertainty propagation and error control: As new tasks/fidelities are integrated, propagating epistemic and aleatoric uncertainty, and ensuring this uncertainty reflects true limitations of the fusion, remains difficult, especially when sampling resources are highly constrained.
• Negative transfer: When the tasks or fidelities are not sufficiently correlated, information fusion can degrade rather than improve predictive performance. Approaches that learn the amount of sharing (e.g., gating networks, hierarchical clustering, latent variable embeddings) help mitigate but do not eliminate this risk.

The field remains highly active, with research extending to more general adaptive sampling, robust non-hierarchical multitask fusion, scalable physics-informed surrogates, and unified frameworks combining deterministic and Bayesian modules (Yi et al., 21 Jul 2024, Zhou et al., 14 Apr 2025, Howard et al., 18 Oct 2024).

7. Outlook and Future Directions

Ongoing research trends and open directions include:

• Automated modular fusion: Automated construction of multitask/multifidelity models from pre-trained single-task sources via layer decomposition and Transformer-inspired knowledge fusion modules (Zhou et al., 14 Apr 2025) is lowering the barrier to deploying robust models across heterogeneous tasks and data sources.
• Flexible task/fidelity coupling: Non-hierarchical latent embedding frameworks (e.g., LVGP (Chen et al., 2023)) and conditionally independent latent variable models (Wu et al., 2022) are broadening applicability to complex, nonaligned datasets and accelerating adaptive sampling schemes.
• End-to-end uncertainty quantification: End-to-end Bayesian inference over both fusion parameters and model components (via variational inference or MCMC) is further improving trustworthiness in safety-critical and scientific modeling scenarios (Cutajar et al., 2019, Giannoukou et al., 14 Jul 2025).
• Physics-guided hybridization: Integration of physics-based components, interface learning, and data-driven correction maps promises more trustworthy and generalizable digital twins for real-time monitoring, control, and prediction (San et al., 2021).

A plausible implication is that the convergence between multifidelity, multitask, and active learning research will yield even more adaptive, robust, and cost-efficient predictive frameworks across science and engineering domains. As data landscapes grow more heterogeneous and resource trade-offs remain acute, the rigorous treatment of relatedness—by both statistical and mechanistic means—is poised to remain central to predictive modeling practice.