Hybrid Multi-Fidelity Models

• Hybrid multi-fidelity models are frameworks that combine physics-based low-fidelity approximations with high-fidelity data-driven corrections to balance simulation efficiency and accuracy.
• They employ additive correction and neural-augmented methods, where models like SIREN networks refine coarse predictions by learning corrections from limited high-fidelity data.
• These models yield significant speedups and robust control performance by training offline with paired LF and HF trajectories for reinforcement learning applications.

Hybrid multi-fidelity models are frameworks that integrate computational and data-driven models of different fidelities—balancing the accuracy and physical interpretability of high-fidelity (HF) models with the scalability and efficiency of low-fidelity (LF) models. These approaches are foundational in areas where accurate high-fidelity data (e.g., from fine-mesh PDE solvers, experiments, or costly simulations) are prohibitively expensive or scarce, while low-fidelity approximations (e.g., reduced-order models, coarse simulations, empirical surrogates) are abundant but inaccurate in critical regimes. Hybridization leverages the strengths of each fidelity by systematically fusing, correcting, or aligning model outputs, often yielding both order-of-magnitude speedups and substantial gains in prediction accuracy or control performance relative to single-fidelity or purely data-driven methods.

1. Foundations of Hybrid Multi-Fidelity Modeling

Hybrid multi-fidelity frameworks systematically couple models and data from different accuracy levels, often by embedding a “dominant” physics-based (LF) description and augmenting it with corrections derived from data or high-fidelity simulations. Two broad classes are identifiable:

• Additive Correction Models: The low-fidelity physics-based surrogate is corrected by a learned, data-driven model trained on residuals with respect to HF data. This presents as

$f_{MF}(x) = f_{LF}(x) + \Delta(x)$

where $f_{LF}$ is the LF model, and $\Delta(x)$ is a high-fidelity-informed correction (often from a neural network, RBFN, or GP).

• Neural-Augmented Hybrid Models: The LF model output is fed, together with system state and/or actions, into a neural corrector, such as a SIREN with periodic activation, to yield a predictor-corrector structure. For time-evolving systems, the hybrid step is

$$u^{C}_{t+1} = F^L(u_t, a_t; \mu) + f_{\theta}(u^C_t, F^L(u_t,a_t;\mu), a_t; \mu, \theta)$$

ensuring the correction acts on LF-driven trajectories (Sun et al., 8 Apr 2025).

In both cases, training the correction is typically performed offline on paired LF and (limited) HF trajectories, with the resulting hybrid model used as the environment or surrogate for subsequent RL or optimization.

2. Hybrid Model Training and Architecture

The prototypical construction consists of the following steps:

• LF Model Definition: Formulate or select a computationally efficient, possibly coarse or physics-based, LF model. For example, spatially-averaged ODEs or coarse-grid LES surrogates for complex dynamical PDEs.
• HF Data Curation: Collect HF output, either from fine-grid direct numerical simulations, experiments, or detailed simulators.
• Correction Model Parameterization: Define a neural architecture for $f_{\theta}$. SIREN networks are preferred for stiff/chaotic systems due to their ability to capture high-frequency oscillations and sharp transients.
• Offline Training: Minimize a fidelity-consistent loss, usually squared error between the hybrid model’s projection and that of the HF system:

$$L(\theta) = \frac{1}{N T} \sum_{i=0}^{T-1} \sum_{j=1}^n \| u^{C}_{i+1} - G(u^{H}_{i+1}) \|^2$$

where $G$ maps HF states onto the LF state space.

After correction training, the hybrid model is “frozen” and used as a fast, differentiable proxy for policy optimization, control reinforcement learning, or design search.

3. Hybrid Multi-Fidelity Reinforcement Learning (MFRL)

When the operational goal is control or sequential decision-making under uncertainty, the hybrid model serves as a closed-loop environment for deep RL agents. The complete MFRL pipeline consists of:

• Environment Rollouts: The RL agent (e.g., TD3 actor-critic) interacts with the differentiable hybrid environment, enabling high-frequency, computationally cheap queries approximating HF dynamics without directly invoking HF solvers post-training.
• Spectral-Domain Rewards: For chaotic or highly structured systems, rewards are often defined in the frequency or spectral domain, using metrics such as summed peak energies ($\ell_1$), squared energies ($\ell_2$), and spread of dominant frequencies ($\Delta_\omega$). This approach supports robust, goal-aligned RL even when pointwise state errors are difficult to penalize effectively (e.g., in plasma or turbulence control).
• Control Policy Optimization: Standard RL loop with experience replay, delayed policy (actor) updates, and Bellman error minimization, all performed using the hybrid surrogate.

The MFRL setup achieves near-HF performance for control objectives at a fraction of the HF simulation cost, and outperforms both single-fidelity surrogates and purely data-driven (ML-only) baselines by significant margins in relevant metrics (Sun et al., 8 Apr 2025).

4. Specific Algorithmic Outline and Quantitative Results

Algorithmic Structure

Offline Correction Learning

1. Sample multi-fidelity (LF, limited HF) state-action trajectories.
2. For each step: $u' = F^L(u, a; \mu)$; $u^C = u' + f_{\theta}(...)$; compute projected-state loss to HF.
3. Update correction parameters $\theta$ by gradient descent.

RL Training in Hybrid Environment

1. Initialize policy and critic networks.
2. At each rollout step: a. Compute $a_t = \pi(u_t) +$ noise. b. Advance system as $u_{t+1}$ via the hybrid model. c. Compute reward $r_t$ using spectral/frequency-based metrics. d. Store transitions in replay buffer.
3. Periodically update actor/critic networks and target networks as per TD3 or similar algorithms.

Representative Metrics (Plasma SRS/Burgers)

Model	$\ell_1$	$\ell_2$	$\Delta_\omega$	KL-div	SMSE
HF DNS	2.13	1.24	0.91	0	0
LF ODE	36.2	18.4	1.62	0.16	4.88
Hybrid ODE	2.60	1.30	0.96	9.1e-3	0.76

For plasma SRS control, the hybrid achieves near-HF performance, with spectral content (and therefore physical behaviors) faithfully matched, and massive reduction in KL divergence and error metrics relative to the LF baseline.

5. Practical Implementation Considerations

Key attributes underlying high efficacy of hybrid multi-fidelity models in complex system control include:

• Differentiability: The hybrid models, as composition of LF physics and neural (SIREN) correctors, are fully differentiable, supporting gradient-based training and compatibility with modern RL/optimization frameworks.
• Zero HF Cost at Deployment: Once trained, the hybrid model incurs no additional HF simulation calls. All environment rollouts, policy updates, and uncertainty propagation rely only on the offline-trained surrogate.
• Robustness to Chaotic/Nonlinear Regimes: Neural correctors with global smooth activation spaces (e.g., SIREN) effectively fit highly oscillatory or multi-scale error dynamics that arise when surrogating strongly nonlinear PDEs, and thus outperform baseline ML methods in stiff or turbulent testbeds.

6. Extensions and Comparative Contexts

Hybrid multi-fidelity frameworks as described in (Sun et al., 8 Apr 2025) have resonance with numerous complementary paradigms:

• Data-Driven/Model-Based Switching: Some frameworks alternate constitutive models (e.g., poroelasticity models with k-d tree nearest-neighbor search for either stress or Darcy flow depending on data density) (Bahmani et al., 2020), using fidelity-adaptive cost functions.
• Operator Correction in PDE Models: DNS/LES surrogates or reduced basis models with neural or GP corrections are structurally aligned with hybrid predictor-corrector surrogates.
• Multi-fidelity RL Beyond Hierarchies: Adaptive methods (e.g., ALPHA) select among non-hierarchical fidelity models based on local alignment of policy outputs, dynamically balancing exploration and exploitation while preventing overreliance on poorly correlated LF surrogates (Agrawal et al., 16 Nov 2024).
• High-dimensional and Non-nested Generalizations: Deep neural and hierarchical frameworks incorporating manifold alignment, autoencoder fusion, and latent variable models extend these concepts to problems with heterogeneous discretizations, modalities, or dimensionalities.

7. Limitations and Future Directions

Hybrid multi-fidelity models provide a systematic path to reduced computational cost and enhanced model fidelity, but notable constraints remain:

• Correction Model Limitations: SIREN or neural correction networks, while expressive, may require significant HF data for adequate coverage in highly nonlinear regions.
• Extrapolation Risk: The fidelity gap between the LF model and the HF ground truth bounds the achievable accuracy; systematic bias or unmodeled dynamics unmet by the correction may persist.
• Reward Design: Choice of reward function (e.g., spectral metrics vs. pointwise errors) fundamentally impacts RL convergence and control performance, especially in chaotic systems.

Active research foci include more principled uncertainty quantification, adaptive hybridization with dynamic model switching, and generalization across multi-domain settings where input mappings across fidelity levels may not be trivial.

---

References:

• Multi-fidelity RL control via hybrid SIREN-corrected surrogates: "Multi-fidelity Reinforcement Learning Control for Complex Dynamical Systems" (Sun et al., 8 Apr 2025)
• Data-driven/model-based switching for poroelasticity: "An accelerated hybrid data-driven/model-based approach for poroelasticity problems with multi-fidelity multi-physics data" (Bahmani et al., 2020)
• Non-hierarchical adaptive multi-fidelity RL: "Adaptive Learning of Design Strategies over Non-Hierarchical Multi-Fidelity Models via Policy Alignment" (Agrawal et al., 16 Nov 2024)