Multi-Fidelity Residual Neural Processes

• Multi-Fidelity Residual Neural Processes (MFRNP) are surrogate models that explicitly model the residual between aggregated lower-fidelity predictions and high-fidelity data to enhance accuracy.
• They employ a cascade of Neural Processes with fidelity-specific encoder-decoder networks that aggregate lower-level outputs into a refined high-fidelity estimate.
• MFRNP demonstrates superior scalability, robust out-of-distribution performance, and significant error reduction in applications ranging from PDE benchmarks to climate modeling.

Multi-Fidelity Residual Neural Processes (MFRNP) are a class of surrogate modeling frameworks that leverage data from simulators or data sources of varying fidelity to achieve efficient and scalable approximation of complex high-fidelity responses. MFRNP explicitly models the residuals between aggregated lower-fidelity predictions and the ground-truth highest-fidelity observations, introducing this residual as a structured correction via neural process surrogates. The approach addresses key limitations of Gaussian process (GP) and traditional neural-process-based strategies, offering improved high-dimensional scalability and robust extrapolation in out-of-distribution (OOD) regimes (Niu et al., 2024).

1. Multi-Fidelity Surrogate Problem and Residual Formulation

In multi-fidelity settings, one considers $K$ simulators or data sources $\{f_k\}_{k=1}^K$ of increasing fidelity and computational cost ($c_1 < c_2 < \cdots < c_K$), all mapping inputs $x \in \mathbb{R}^{d_x}$ to outputs $y \in \mathbb{R}^{d_y}$. The practical objective is to learn a surrogate $\hat f_K(x)$ for the most accurate simulator $f_K$ by exploiting all available data $(x_{k,n}, y_{k,n})$ from each fidelity level, especially when high-fidelity samples are scarce.

MFRNP extends the residual paradigm introduced by Davis, Motamed, and Tempone—where a residual function $r(x)$ models the discrepancy $f_H(x) - f_L(x)$ between low- and high-fidelity models (Davis et al., 2023)—to general $\{f_k\}_{k=1}^K$0-fidelity settings, encoding both the reconstructed aggregate of lower-fidelity predictors and their remaining error as a learnable residual. This hierarchy is formalized as:

$\{f_k\}_{k=1}^K$1

where $\{f_k\}_{k=1}^K$2 are decoded NP predictions from lower fidelities, interpolated to the resolution of the top-fidelity output (Niu et al., 2024).

2. Neural Process Architecture and Aggregation Mechanism

At the core of MFRNP is a cascade of Neural Processes (NPs) with fidelity-specific encoder and decoder networks. For each $\{f_k\}_{k=1}^K$3, the NP at fidelity $\{f_k\}_{k=1}^K$4 utilizes

• a permutation-invariant encoder $\{f_k\}_{k=1}^K$5 mapping context data to a Gaussian latent variable $\{f_k\}_{k=1}^K$6,
• a decoder $\{f_k\}_{k=1}^K$7 producing predictive distributions for target inputs.

Unlike prior methods that only share representations at the encoder level, MFRNP propagates lower-fidelity decoder outputs forward by aggregating their decoded predictions. This aggregation is used to form $\{f_k\}_{k=1}^K$8, over which the residual at the highest fidelity is modeled as a separate NP:

• The surrogate for $\{f_k\}_{k=1}^K$9 (residual) uses its own encoder and decoder $c_1 < c_2 < \cdots < c_K$0, with training targets formed as $c_1 < c_2 < \cdots < c_K$1 for each available high-fidelity sample.
• At inference, MFRNP first computes $c_1 < c_2 < \cdots < c_K$2 using ancestral sampling across lower-fidelity NPs, then applies the highest-fidelity NP to estimate $c_1 < c_2 < \cdots < c_K$3, producing the final output $c_1 < c_2 < \cdots < c_K$4 (Niu et al., 2024).

This architectural choice enforces cross-fidelity alignment, as the optimization of the residual NP backpropagates into lower-fidelity decoders, ensuring they yield aggregations optimally informative for the correction.

3. Loss Function and Training Objective

MFRNP training optimizes a tailored variational Evidence Lower Bound (ELBO) for the entire multi-fidelity cascade:

• For fidelities $c_1 < c_2 < \cdots < c_K$5, the standard NP ELBO is computed over all context/target splits:

$c_1 < c_2 < \cdots < c_K$6

• For the residual process at fidelity $c_1 < c_2 < \cdots < c_K$7, an analogous ELBO is maximized over the pseudo-dataset $c_1 < c_2 < \cdots < c_K$8, conditioning on the aggregation from all lower-fidelity decoders.

The overall MFRNP loss for parameters $c_1 < c_2 < \cdots < c_K$9 can be written as:

$x \in \mathbb{R}^{d_x}$0

where $x \in \mathbb{R}^{d_x}$1 sums over lower-fidelity ELBOs and $x \in \mathbb{R}^{d_x}$2 is the residual's ELBO (Niu et al., 2024).

4. Scalability, Computational Benefits, and Out-of-Distribution Robustness

Unlike autoregressive Gaussian process approaches, which scale cubically in sample size and suffer in high dimensions, MFRNP inherits the favorable linear complexity of neural processes with respect to the sample set and input dimension. Decoder and encoder operations per fidelity are independent small neural networks (typically MLPs or CNNs).

The inclusion of lower-fidelity decoders in the aggregated prediction, and the explicit modeling of the residual, yield:

• Efficient use of multi-fidelity data: increasing the number of fidelity levels often lowers error, especially when high-fidelity coverage is limited.
• Robustness to out-of-distribution inputs: explicit residual learning enables MFRNP to extrapolate correction terms beyond the sparse domain of $x \in \mathbb{R}^{d_x}$3 training data, as shown by order-of-magnitude degradation in OOD error when replacing aggregation with a latent hierarchy (ablation MFRNP-H) (Niu et al., 2024).

5. Empirical Performance in Scientific and Real-World Applications

Key experiments demonstrate the advantage of MFRNP:

• On steady-state heat, Poisson, and Navier–Stokes PDE benchmarks (with up to 5 fidelities), MFRNP reduces normalized RMSE by approximately 90% relative to competing methods (e.g., DMF, NARGP, MFHNP, D-MFD).
• In global climate modeling (up to 9 fidelities, $x \in \mathbb{R}^{d_x}$4 grid, CMIP6+ERA5), MFRNP consistently outperforms DeepESD CNN, NNGPR, MFHNP, and D-MFD, achieving lower nRMSE both in- and out-of-distribution, and demonstrates stable error behavior in long-term projections (Niu et al., 2024).
• In robotics state estimation under real-time constraints, the Multi-Fidelity Residual Physics-Informed Neural Process (MFR-PINP) adapts the MFRNP paradigm to physics-informed settings and residual learning, supported by stable sub-millisecond inference on embedded hardware and outperforming transformer-based Deep Kalman Filters in RMSE and negative log-likelihood (Hunter et al., 11 Nov 2025).

6. Implementation Considerations and Domain-Specific Extensions

Implementation parameters are domain dependent but adhere to the following guidelines:

• Latent dimension $x \in \mathbb{R}^{d_x}$5: ranges from 32 (PDEs), to 128 (fluid), to 512 (climate and multi-fidelity tasks with many input features).
• Architecture: encoders and decoders are task-adapted (MLP for vector data, CNN for spatial fields); context sets use 20–25% of each fidelity's data.
• Optimization: Adam with initial learning rate $x \in \mathbb{R}^{d_x}$6 and decay; early stopping is used for convergence.
• Inference: two-pass ancestral sampling—lower-fidelity decoders generate $x \in \mathbb{R}^{d_x}$7, then the residual NP predicts $x \in \mathbb{R}^{d_x}$8 (Niu et al., 2024).
• For uncertainty quantification, conformal calibration can be layered over NP-predicted uncertainty, ensuring empirically valid coverage intervals (Hunter et al., 11 Nov 2025).

Physics-informed extensions incorporate constraints from underlying dynamical models into the residual-predicting NP, with a weak regularization effect achieved by training the residual decoder to predict differences between a physics-based prior and observed dynamics (Hunter et al., 11 Nov 2025).

7. Relation to Residual Multi-Fidelity Neural Networks and Theoretical Motivation

MFRNP generalizes the two-stage Residual Multi-Fidelity Neural Network (RMFNN) framework. In RMFNN, a compact ResNet first learns the (small-magnitude) non-linear residual $x \in \mathbb{R}^{d_x}$9 with minimal high-fidelity data, which is used to generate synthetic high-fidelity samples for a deeper final network. This reduces training cost while achieving a target $y \in \mathbb{R}^{d_y}$0 accuracy with provably reduced network complexity when the residual norm is small (Davis et al., 2023).

MFRNP extends this to multiple fidelities, neural process surrogates, and probabilistic inference in high dimensions, maintaining the core principle that residual learning enables efficient allocation of network and data resources. Theoretical bounds from approximation theory for ReLU networks motivate this residual-first hierarchy, as network complexity scales favorably with the residual's uniform (or $y \in \mathbb{R}^{d_y}$1) norm (Davis et al., 2023).

Summary Table: Core Components of MFRNP

Component	Functionality	Cited Work
Lower-fidelity NPs	Surrogate lower-cost simulators	(Niu et al., 2024)
Aggregation $y \in \mathbb{R}^{d_y}$2	Average/interpolated prediction from NPs	(Niu et al., 2024)
Residual NP	Learns $y \in \mathbb{R}^{d_y}$3 correction	(Niu et al., 2024)
Joint ELBO training	Backpropagates residual gradients to all NPs	(Niu et al., 2024)
Two-pass sampling	Aggregation then residual prediction	(Niu et al., 2024)

The MFRNP framework introduces a principled way of synthesizing information across multiple fidelities through explicit residual modeling, leading to scalable, accurate, and robust surrogates for complex high-fidelity systems. This methodology is applicable across scientific computing, climate science, and real-time estimation domains (Davis et al., 2023, Niu et al., 2024, Hunter et al., 11 Nov 2025).