Bayesian & Uncertainty-Aware PINNs

• Bayesian and uncertainty-aware PINNs are computational frameworks that fuse probabilistic modeling with physical laws to solve PDEs and inverse problems.
• They leverage Bayesian inference, variational methods, and ensemble techniques to rigorously quantify both aleatoric and epistemic uncertainties.
• These approaches yield well-calibrated credible intervals and uncertainty diagnostics, enabling robust predictions even in data-sparse or noisy regimes.

Bayesian and uncertainty-aware physics-informed neural networks (PINNs) are a class of scientific machine learning frameworks that integrate probabilistic modeling with deep neural representations constrained by physical laws, enabling not only the solution of partial differential equations (PDEs) and inverse problems, but also the rigorous quantification and decomposition of both aleatoric and epistemic uncertainties. These methods leverage Bayesian inference, variational techniques, evidence maximization, ensemble diversity, and nonparametric or fiducial approaches to produce well-calibrated credible intervals and uncertainty diagnostics for predictions, parameters, and latent physical fields—even in data-sparse, noisy, or ill-posed regimes. The following sections survey the established methodologies, algorithmic advances, theoretical foundations, and empirical benchmarks of Bayesian and uncertainty-aware PINN frameworks, with particular emphasis on the computational techniques and uncertainty quantification (UQ) metrics validated in contemporary research.

1. Bayesian Formulation and Uncertainty Decomposition

A canonical Bayesian PINN introduces probability distributions over both the neural network weights $\theta$ and any unknown latent fields, parameters, or boundary conditions. A Gaussian or sparsity-inducing prior $p(\theta)$ is placed on the weights, and noisy observations (data, boundary values, collocation points) are modeled with likelihoods parametrized by the network output, often assuming additive Gaussian noise with standard deviations reflecting measurement fidelity (Yang et al., 2020). The resulting posterior,

$$p(\theta \mid \mathcal{D}) \propto p(\mathcal{D} \mid \theta) p(\theta),$$

enables the propagation of uncertainty from both data and model uncertainty into posterior predictive distributions for the PDE solution and any inferred physical fields or parameters. Aleatoric uncertainty arises from noise in the data or measurement models, quantified through likelihood variance; epistemic uncertainty is captured by the spread in $p(\theta \mid \mathcal{D})$ or functional posteriors (Ramirez et al., 7 Jan 2026). Algorithmic realizations include full Bayesian neural networks (Yang et al., 2020), hierarchical modeling over fields and weights for high-dimensional inverse problems (Mohammad-Djafari et al., 2 Dec 2025, Mohammad-Djafari, 4 Feb 2026), and explicit inclusion of prior information on solution regularity or correlational structure.

2. Posterior Inference: HMC, Variational Approximations, and Laplace Methods

Several inference strategies are prevalent for approximating the intractable PINN posteriors:

• Hamiltonian Monte Carlo (HMC): HMC simulates Hamiltonian dynamics in parameter space to obtain asymptotically exact posterior samples. It is robust for multimodal, high-dimensional neural posteriors (e.g., $d_\theta > 10^3$) but incurs significant computational cost (Yang et al., 2020, Imanov, 1 Feb 2026). HMC maintains calibrated uncertainty intervals under large data noise and complex PDEs.
• Variational Inference (VI): Mean-field Gaussian variational surrogates $q(\theta)$ are trained to minimize KL divergence to the true posterior. This approach is much faster but tends to underestimate uncertainty, especially in multimodal or stiff problems (Graf et al., 2022). Variational posteriors enable efficient UQ on large-scale models and are suitable for online or high-throughput training.
• Laplace Approximation: The Laplace method approximates the posterior by a Gaussian centered at the MAP (maximum a posteriori) solution and posterior covariance given by the inverse Hessian of the loss. This method supports evidence maximization for hyperparameter tuning and efficient UQ but is limited to locally unimodal regions and may underestimate uncertainty in nonlinear settings (Graczyk et al., 2023).
• Monte Carlo Dropout: Dropout at inference time approximates Bayesian model averaging, yielding empirical uncertainty bands, but these intervals are highly sensitive to the dropout rate and may not distinguish between aleatoric and epistemic sources (Yang et al., 2020, Nair et al., 25 Mar 2025).

3. Error-Aware, Heteroscedastic, and Structured Likelihoods

Accurate UQ in PINNs requires both statistical modeling of data noise and structural error due to model inadequacy:

• Error-Aware B-PINN: For linear ODEs and certain PDEs equipped with rigorous error bounds, the solution error can be explicitly estimated from the physical residuals. These bounds are mapped into heteroscedastic variance models in the surrogate BNN likelihood, yielding predictive bands that realistically widen in regions of high physical error or extrapolation (Graf et al., 2022, Flores et al., 9 May 2025).
• Heteroscedastic Likelihoods: Outputs for both mean and variance are predicted by the PINN, allowing them to adapt to input features, uncertainty in physical conditions, or nonuniform data noise (Ramirez et al., 7 Jan 2026).
• Residual-Noise Modeling: The variance of the physics residual is treated as an unknown and either estimated from data or assigned an evidence-based prior; this prevents the underestimation of epistemic uncertainty (Tan et al., 18 Sep 2025).

4. Ensembles, Repulsion, and Posterior Diversity

Ensemble methods augment Bayesian or variational approaches by explicitly training multiple models:

• Standard Deep Ensembles: Multiple independently-initialized PINNs approximate the uncertainty by the spread in their predictions; however, this ensemble typically collapses to a set of closely grouped MAP solutions, failing to capture true posterior diversity (Pilar et al., 22 May 2025).
• Repulsive Ensembles (RE-PINN): Inclusion of a repulsive density term in the loss or through kernel-density estimation in function/parameter space ensures that the ensemble’s empirical distribution converges to the correct Bayesian posterior in the limit of infinite members (Pilar et al., 22 May 2025). Empirically, repulsive ensembles deliver calibrated uncertainty bands and better coverage than undiversified ensembles.
• Latent Variable Models and Neural Operators: Approaches such as LVM-GP interpolate between neural feature encoders and functional Gaussian process priors, enabling structured modeling of epistemic uncertainty and efficient UQ with limited samples (Feng et al., 30 Jul 2025).

5. Domain Decomposition, Multi-Fidelity, and Large-Scale Bayesian PINNs

Advanced PINN UQ methods scale up to multi-scale or high-dimensional problems by exploiting locality and data hierarchy:

• Domain Decomposition ($PINN): The joint Bayesian posterior over a global PDE solution is factorized into local posterior distributions for each subdomain, constrained by interface continuity and flux-matching. This hierarchy enables modular parallel training and faithful propagation of local uncertainty into global credible intervals (Figueres et al., 26 Apr 2025).
• Multi-Fidelity Bayesian PINNs: By modeling the relationship between multiple fidelities (e.g., LF and HF simulations), the Bayesian PINN utilizes an adaptive gating network to interpolate between linear and nonlinear corrections. HMC-based posterior sampling propagates both data and model uncertainties through the multi-fidelity architecture, yielding pointwise credible intervals and rigorous UQ metrics (Imanov, 1 Feb 2026).

6. Alternative and Distribution-Free UQ Paradigms

Beyond Bayesian and ensemble approaches, several alternative uncertainty-aware PINN paradigms have recently emerged:

• Conformalized PINNs (C-PINN): Use of conformal prediction intervals, constructed by calibrating PINN residuals on held-out data, yields finite-sample, distribution-free coverage guarantees. These intervals adapt to PINN fit quality and are minimally sensitive to noise-model misspecification (Podina et al., 2024).
• Evidential Deep Learning within PINNs (E-PINN): A Normal-Inverse-Gamma evidential prior is imposed on output mean and variance, providing analytic Student-t predictive distributions and calibrated confidence intervals, often outperforming both standard B-PINNs and ensembles in empirical coverage (Tan et al., 18 Sep 2025).
• Extended Fiducial Inference for PINNs (EFI-PINN): Fiducial methods propagate uncertainty from data noise through a learned inverse mapping (narrow-neck hyper-network) to obtain asymptotically honest confidence bands for PINN solutions and parameters, without requiring any subjective prior (Shih et al., 25 May 2025).
• Randomized PINNs (rPINN): Randomizing each term in the PINN loss to simulate the stochastic optimization problem yields a collection of solutions whose empirical distribution approximates the Bayesian posterior without the convergence and mixing pathologies of high-dimensional HMC (Zong et al., 2024).

7. Applications, Benchmarks, and Comparative Performance

Bayesian and uncertainty-aware PINNs have demonstrated empirical superiority over deterministic and ensemble-only methods across a variety of benchmarks:

• Forward and Inverse Problems: Benchmarks include parametric PDEs, nonlinear ODEs, heat/wave/Burgers' equations, 2D reaction-diffusion, and direct/indirect imaging inverse problems (Yang et al., 2020, Raj et al., 18 Jan 2025, Mohammad-Djafari et al., 2 Dec 2025, Mohammad-Djafari, 4 Feb 2026).
• Metrics: Evaluations focus on empirical credible interval coverage, negative log-likelihood (NLL), continuous ranked probability score (CRPS), miscalibration area, interval sharpness, RMSE, PSNR/SSIM for imaging, and expected calibration error. For instance, in forward PDE tests, B-PINN-HMC coverage consistently exceeded 90% under heavy noise, while variational and dropout PINNs often yielded either under- or overconfident intervals (Graf et al., 2022, Ramirez et al., 7 Jan 2026, Tan et al., 18 Sep 2025).
• Scalability: Modular domain decomposition, multi-fidelity learning, and efficient inference algorithms (HMC, repulsive ensembles, evidential/statistical surrogates) facilitate tractable UQ on large-scale or high-dimensional applications (Figueres et al., 26 Apr 2025, Imanov, 1 Feb 2026).
• Interpretability: Posterior variance maps inform on spatial or parametric regions of high epistemic or solution error, facilitating risk-aware decision making as exemplified by uncertainty maps in transformer asset management and infrared imaging (Mohammad-Djafari et al., 2 Dec 2025, Ramirez et al., 7 Jan 2026).

8. Theoretical Developments, Limitations, and Future Directions

Ongoing research targets rigorous theory, computational scalability, and extensions to broader classes of uncertainty:

• Total Error Bounds: Explicit mapping from residuals to solution error using a priori bounds is central to calibrated UQ but largely restricted to linear or weakly nonlinear PDEs; extension to strongly nonlinear systems is an open challenge (Flores et al., 9 May 2025, Graf et al., 2022).
• Calibration and Miscoverage: Standard B-PINN and dropout methods are sensitive to prior and noise-scale choice; EFI and conformal UQ approaches circumvent these limitations and deliver finite-sample exactness (Podina et al., 2024, Shih et al., 25 May 2025).
• Posterior Structural Modeling: Incorporation of functional, spatially structured priors (e.g., GP, Karhunen–Loève expansions) and non-Gaussian variational families addresses limitations of mean-field or local Laplace approaches (Feng et al., 30 Jul 2025, Graczyk et al., 2023).
• Computational Cost: HMC provides the gold standard in UQ but is computationally demanding; variational, ensemble, and randomized strategies trade off speed for exactness, with current research focusing on algorithmic acceleration, parallelization, and richer approximations (Zong et al., 2024, Pilar et al., 22 May 2025).
• Active Learning and Sensor Placement: Dropout uncertainty guides active learning for experimental design, e.g., adaptive sensor selection to optimally reduce total uncertainty in multi-modal or high-dimensional parameter spaces (Zhang et al., 2018).

A plausible implication is that Bayesian and uncertainty-aware PINNs furnish the foundational toolkit for uncertainty-aware physical simulation and inverse reasoning, enabling practitioners to compute interpretable, quantitatively calibrated error bars that inform model deployment and scientific inference in data-limited, high-stakes applications.