Two-Stage Physics-Informed Bayesian Framework

• The framework is a two-stage approach that decouples physics-informed model construction from Bayesian inference to achieve efficient uncertainty quantification.
• Stage 1 leverages physics-constrained surrogates (e.g., PINNs, GPs, PI-GANs) using low-fidelity data, while Stage 2 applies rigorous Bayesian sampling with high-fidelity observations.
• The method enables scalable training with explicit epistemic and aleatoric uncertainty quantification, addressing inverse problems and complex parametric PDEs.

A two-stage physics-informed Bayesian framework is a hierarchical modeling and inference strategy that explicitly combines mechanistic physical laws with probabilistic machine learning (typically neural networks or Gaussian processes), decoupling the learning and uncertainty quantification phases for computational efficiency and robust epistemic calibration. The two-stage principle underlies recent advances in multi-fidelity surrogate modeling, model-based UQ, and adaptive scientific machine learning applied to inverse problems, parametric PDEs, and data assimilation. Typical instantiations proceed by first constructing an informative, physics-constrained model using historical data or low-fidelity simulations, followed by rigorous Bayesian inference (via Hamiltonian Monte Carlo or variational methods) conditional on partial, often noisy, high-fidelity or real-world observations. This division of labor enables both scalable training and principled quantification of both epistemic and aleatoric uncertainties.

1. Core Structure and Problem Formulation

The general formalism comprises a parametric physical system (such as a PDE)

$$\mathcal N[u(x,t;\mu)] = f(x,t;\mu), \qquad (x,t)\in\Omega\times[0,T],~\mu\in\mathcal P\subset\mathbb R^d,$$

with given boundary and initial conditions. Here $u$ is the solution field, $\mu$ encodes parametric physical coefficients, and $f$ is the source or forcing term. Realistic applications frequently face sparse, noisy, and multi-fidelity data scenarios, where obtaining high-fidelity solutions $\tilde u^{HF}$ is expensive. Physics-informed machine learning frameworks aim to approximate $u$ flexibly and extract latent parameters, but traditional end-to-end Bayesian learning is computationally prohibitive due to high-dimensional parameter spaces, especially for deep NN surrogates.

Two-stage frameworks address this by separating model construction (Stage 1) from posterior inference (Stage 2):

• Stage 1: Learn a model or prior that encodes physics using low-fidelity data, historical observations, or simulation outputs, potentially aided by flexible representations (e.g., PINNs, deep GPs, PI-GANs, operator networks), and regularized by physics loss terms.
• Stage 2: Given limited high-fidelity or experimental data, perform Bayesian inference—using HMC, variational Bayes, or similar—either in the low-dimensional parameter subspace, the final-layer (Bayesian last-layer NNs), or the latent space of generative models.

This approach appears in several settings:

• Multi-fidelity physics-informed PINNs with Bayesian UQ and adaptive residuals (Imanov, 1 Feb 2026)
• Hybrid PIML–BNN with two-stage variational/Monte Carlo UQ (Oddiraju et al., 23 Jun 2025)
• Physics-based DKL surrogate with GP and HMC stages for parameter estimation (Yan et al., 17 Sep 2025)
• Probit-warped GP updating with physics-based fragility in hazard models (Braik et al., 19 Jan 2026)
• Deep generative latent prior followed by HMC in latent space (PI-GANs) (Meng et al., 2021)

2. Stage 1: Physics-Informed Model/Prior Construction

Neural-network, GP, or GAN-based architectures are trained to encode physical structure in the solution or response function. Several technical mechanisms are documented:

• PINNs (Physics-Informed Neural Networks) where the solution surrogate $u_{LF}(x,t;\mu)$ is parameterized as $\mathcal N_{LF}(x,t,\mu;\theta_{LF})$ and trained via a hybrid loss comprising both data fit and the squared PDE residual at collocation points (Imanov, 1 Feb 2026).
• Functional Priors and Generative Models: Physics-informed GANs (PI-GANs) learn distributions over fields conditioned on data or simulated physics samples, using adversarial objectives regularized by automatic-differentiation-derived PDE residuals. The generator becomes a probabilistic prior over solution fields (Meng et al., 2021).
• Physics-Based Deep Kernel Learning: A composition of a neural network feature extractor $\phi(\cdot;w)$ with a base kernel $k_{base}$ creates a deep kernel $u$0, and the GP surrogate $u$1 is trained under data, physics residual, and marginal likelihood penalties (Yan et al., 17 Sep 2025).
• Physics-Informed GPs: For linear parametric models, a multi-output GP prior is constructed by analytically propagating the physical operator $u$2 through the covariance, yielding joint GPs for both the field and its PDE residual, e.g., $u$3 (Spitieris et al., 2022).

At the end of Stage 1, the physics-regularized model or functional prior is fit to all available data, possibly yielding MAP estimates, posterior means and variances, or deterministically fixed weights for NNs or kernel features.

3. Stage 2: Bayesian Inference and Posterior Sampling

In Stage 2, the learned model from Stage 1 forms the backbone for Bayesian inference of parameters, predictions, or latent representations using sparse, higher-fidelity observations. This is accomplished via rigorous sampling or variational methods:

• Full-Parameter, Low-Dimensional, and Bayesian Last-Layer Inference: Depending on the computational cost and identifiability, Bayesian inference is conducted over all parameters (e.g., GP kernel hyperparameters and physics parameters $u$4, $u$5 in (Yan et al., 17 Sep 2025)), or restricted to a low-dimensional kinetic subspace (e.g., $u$6 for Gompertz ODEs in tumor modeling (Kong et al., 13 May 2026)), or to the last-layer weights of a neural transfer network (BNN) with early layers fixed (Oddiraju et al., 23 Jun 2025).
• Hamiltonian Monte Carlo (HMC): This is the canonical approach for exploring the posterior, especially in moderate-to-high dimensionality, with leapfrog integration and adaptive step sizes. The potential energy is $u$7, and the Hamiltonian is augmented with a kinetic term for momentum variables (Imanov, 1 Feb 2026, Yan et al., 17 Sep 2025).
• Variational Approximations: For neural-network settings, the Bayesian last-layer strategy employs a variational posterior $u$8 (Gaussian for weights) and minimizes the MSE plus evidence lower bound (ELBO) (Oddiraju et al., 23 Jun 2025).

Stage 2 yields posterior samples or credible bands for parameters and predictions, facilitating uncertainty quantification, model selection, and epistemic risk analysis.

4. Multi-Fidelity and Data Assimilation Extensions

Multi-fidelity settings exploit abundant, cheap low-fidelity data and limited, costly high-fidelity observations. The hierarchical network (e.g., MF-BPINN) constructs a multi-fidelity solution as

$u$9

where $\mu$0 is a learnable gating function balancing linear and nonlinear corrections between fidelities (Imanov, 1 Feb 2026).

In probabilistic fragility modeling under hazard scenarios, stagewise updating proceeds from physics-based log-normal fragility priors to batchwise assimilations of multi-source probabilistic observations. Local Beta–Bernoulli updates with source-dependent fidelity weights are moment-matched back into Gaussian PN representations, then globally assimilated via a probit-warped spatial GP for region-wide inference, accommodating both epistemic and aleatoric uncertainty (Braik et al., 19 Jan 2026).

In online adaptive scenarios (e.g., Bayesian active learning for ALD pulse prediction), a physics-embedded GP kernel provides denoising and signal extraction in Stage 1, followed by analytic or ML-based parameter recovery in Stage 2 (Navabi et al., 20 Feb 2026).

5. Physics Encodings and Surrogate Design

Encoding physical structure in the learning model is accomplished by:

• Automatic Differentiation of Residuals: Differentiable surrogates (PINNs, DeepONet) support direct computation of PDE residuals for training and as explicit likelihood terms in the Bayesian posterior (Meng et al., 2021, Oddiraju et al., 23 Jun 2025).
• Operator-Constrained Covariances: Linear operators are analytically propagated through kernel covariances for GPs (with or without model discrepancy terms), maintaining closed-form multi-output covariance structure (Spitieris et al., 2022).
• Hybrid blocks in PIML: Combination of learned (NN or BNN) transformations with parametric, closed-form physics blocks (e.g., aerodynamic coefficients in aircraft models) allows differentiable end-to-end pipelines with explicit physical interpretability (Oddiraju et al., 23 Jun 2025).
• Domain Warping and Heteroscedasticity: Fragility and probability-of-exceedance fields are parameterized via probit or logit transforms, enabling warping between Gaussian and bounded (0,1) spaces, and heteroscedastic likelihoods in the spatial GP (Braik et al., 19 Jan 2026).
• Model Discrepancy and Misspecification: Discrepancy GPs are often layered atop physics-constrained surrogates to handle model inadequacy or experimental bias, as in Bayesian calibration of imperfect models (Spitieris et al., 2022).

6. Uncertainty Quantification and Performance Insights

Posterior samples enable evaluation of predictive means, variances, and credible intervals for solution fields or inferred parameters. Several metrics are systematically reported:

• Coverage: Empirical fraction of true test points within predicted credible intervals, with 95% coverage typically targeted (Kong et al., 13 May 2026, Imanov, 1 Feb 2026).
• Calibration: Comparison of nominal vs. empirical coverage (ECE), proper scoring rules, and width of credible intervals as a function of domain and data density (Braik et al., 19 Jan 2026).
• Pointwise and Integrated RMSE: Mean squared error of predictive means vs. ground truth, and relative improvement vs. baseline models such as purely data-driven GPs or PINNs.
• Sample Efficiency and Convergence: Multi-fidelity and physics-informed frameworks achieve accuracy and UQ with orders-of-magnitude fewer high-fidelity evaluations (e.g., ALD pulse identification in ≈5 iterations (Navabi et al., 20 Feb 2026)).
• Epistemic vs. Aleatoric Decomposition: Several frameworks explicitly decompose total predictive variance into parameter uncertainty and data noise (Imanov, 1 Feb 2026).

Numerical studies consistently indicate that two-stage strategies yield (i) robust parameter identification even in high-dimensional PDE problems (Yan et al., 17 Sep 2025), (ii) quantitatively calibrated uncertainty bands with well-controlled over/under-coverage (Imanov, 1 Feb 2026, Kong et al., 13 May 2026), and (iii) adaptability to real-world data with non-idealities, e.g., model mis-specification or noisy, incomplete observations (Spitieris et al., 2022, Braik et al., 19 Jan 2026).

7. Representative Algorithms and Pseudocode

Canonical two-stage algorithms adhere to the following template (example: MF-BPINN (Imanov, 1 Feb 2026)):

1. Pretrain a physics-regularized surrogate on low-fidelity data:
   • Minimize MSE plus residual and boundary penalties.
2. Initialize multi-fidelity correction networks (linear, nonlinear, gating).
3. Joint training (all data, all loss terms) to MAP.
4. HMC sampling for full Bayesian posterior of all parameters.
5. Compute posterior predictive means, variances, and quantile-based credible intervals.

Similar structures are reflected in hybrid PIML-BNNs, physics-based DKL-GPs, and probit-warped spatial GPs for fragility updating. All implement explicit physics encodings, data/physics weighting, and Bayesian sampling or variational posterior estimation over appropriately selected parameter subspaces.

8. Impact and Computational Considerations

The two-stage physics-informed Bayesian paradigm enables tractable UQ and scientific interpretation in otherwise intractable or data-poor regimes. By isolating the costly inference only to physically-meaningful parameter subspaces (or carefully-designed neural latent spaces), the frameworks render previously intractable problems (e.g., high-dim PDE inverse problems, rapid adaptive ALD process optimization, real-time hazard response) computationally feasible.

A plausible implication is that, as the complexity of physical models and data grows (e.g., multi-scale or parameter-rich PDEs, spatio-temporal hazard scenarios), hierarchical two-stage Bayesian methods will continue to dominate physically-constrained machine learning for scalable, uncertainty-aware scientific computing (Imanov, 1 Feb 2026, Yan et al., 17 Sep 2025, Correia et al., 16 Jun 2026).