Gaussian Processes for Uncertainty Quantification

• Gaussian Processes are nonparametric Bayesian models that define a distribution over functions to quantify uncertainty with closed-form predictive mean and variance.
• They serve as efficient surrogates in high-dimensional settings by combining dimensionality reduction techniques with precise uncertainty estimates in applications like wind power dispatch.
• GP frameworks effectively decompose aleatoric and epistemic uncertainties and extend to deep, physics-informed, and domain-specific models for risk-aware decision making.

Gaussian processes (GPs) constitute a central class of nonparametric probabilistic models for uncertainty quantification (UQ) in computational science, engineering, and machine learning. A GP specifies a distribution over functions, allowing closed-form inference for both predictive mean and variance given observed data. This Bayesian framework provides a principled mechanism to propagate uncertainty from random inputs through surrogate models, quantify both epistemic (model) and aleatoric (data) uncertainty, and facilitate risk-aware decision making in high-dimensional, complex systems.

1. Gaussian Process Formulation for UQ

In the standard regression setting, one places a Gaussian-process prior on an unknown function $f:\mathcal{X}\to\mathbb{R}$,

$$f(\mathbf{x}) \sim \mathcal{GP}(m(\mathbf{x}), k(\mathbf{x},\mathbf{x}'))$$

where $m(\mathbf{x})$ is the prior mean and $k(\mathbf{x},\mathbf{x}')$ is a covariance kernel encoding spatial/spectral assumptions. Observed data $\{(\mathbf{x}_i, y_i)\}_{i=1}^n$ are modeled as noisy realizations,

$$y_i = f(\mathbf{x}_i) + \epsilon_i, \quad \epsilon_i \sim \mathcal{N}(0, \sigma_n^2)$$

The posterior predictive distribution at a new $\mathbf{x}_*$ is analytically available: $$\begin{aligned} \mu(\mathbf{x}_*) &= k(\mathbf{x}_*, X) [K + \sigma_n^2 I]^{-1} \mathbf{y} \ \sigma^2(\mathbf{x}_*) &= k(\mathbf{x}_*, \mathbf{x}_*) - k(\mathbf{x}_*, X)[K + \sigma_n^2 I]^{-1} k(X, \mathbf{x}_*) \end{aligned}$$ where $K$ is the Gram matrix on training inputs. This closed-form inference fundamentally enables robust UQ, yielding both a mean estimate and a calibrated pointwise uncertainty quantification for any input location (Li et al., 5 Feb 2025).

2. Surrogate-GP Emulation and Dimensionality Reduction

GPs are empirically indispensable as emulators of expensive solvers, especially in stochastic systems with high-dimensional random fields. For instance, in day-ahead stochastic economic dispatch (SED) under wind power uncertainty, the domain is a spatio-temporal random field with $p=72$ correlated wind variables. Dimension reduction via Karhunen–Loève expansion truncates this to a manageable $p=17$ by preserving >95% variance,

$$X(t) \approx \hat X(t) = \sum_{\ell=1}^p \sqrt{\lambda_\ell} \xi_\ell u_\ell(t)$$

GP surrogates are trained on this reduced subspace, allowing rapid sampling for UQ tasks, such as estimating $\mathbb{E}[Q]$, $\mathrm{Var}[Q]$, and constructing posterior confidence intervals (Hu et al., 2019). This can yield empirically validated speedups (e.g., 80-fold over Monte Carlo) at sub-0.1% bias in critical cost quantities.

For ultra-high-dimensional spaces, built-in dimensionality reduction GP frameworks exploit latent structures. The active subspace GP (AS-GP) embeds a projection $W \in \mathbb{R}^{D \times d}$ ($d \ll D$) as a hyperparameter in the kernel, yielding

$$k_{\mathrm{AS}}(x,x'; W, \theta) = k_0(W^T x, W^T x'; \theta)$$

with $W$ optimized on the Stiefel manifold. Empirical BIC model selection determines the intrinsic $d$ needed for surrogate accuracy (Bilionis et al., 2016).

3. Uncertainty Decomposition: Aleatoric and Epistemic

GP regression separates uncertainty cleanly:

• Aleatoric: The irreducible noise variance, $\sigma_n^2$, intrinsic to measurements or inherent stochasticity.
• Epistemic: The posterior variance of the function itself, $k_{**} - k_*^T(K + \sigma_n^2 I)^{-1} k_*$, quantifying model uncertainty due to limited data.

Total predictive variance is additive,

$$\sigma_*^2 + \sigma_n^2 = \mathrm{Epistemic} + \mathrm{Aleatoric}$$

(Ajirak et al., 7 Sep 2025, Jung et al., 7 Nov 2025). Heteroscedastic GP surrogates further generalize this decomposition, modeling input-dependent noise via a second GP on the log variance. The two-stage approach jointly infers both mean and noise processes, with predictive variance comprising epistemic and aleatoric components, formalized as

$$\mathbb{V}[y(x_*)] = \mathbb{V}_{\mathrm{epistemic}} + \mathbb{V}_{\mathrm{aleatoric}}$$

(Jung et al., 7 Nov 2025).

4. Bayesian Optimization and Risk Estimation with GPs

GPs facilitate global optimization and risk assessment under uncertainty via acquisition functions sensitive to posterior predictive variance. The archetypal Upper Confidence Bound (UCB) criterion is

$$\alpha_{\mathrm{UCB}}(x) = \mu(x) + \kappa \sigma(x)$$

where the exploration coefficient $\kappa$ can be dynamically adapted based on predictive errors, enabling robust exploration-exploitation trade-offs (Gadde, 20 Jul 2025).

Acquisition functions can further penalize local complexity/uncertainty via Hessian-based penalty terms, controlling search toward regions with reliable uncertainty quantification. Regret bounds and convergence rates follow from the information gain of the GP kernel class (e.g., sublinear in $T$, the number of queries, for SE kernels).

In risk estimation tasks, GP surrogates deliver sampling-based or closed-form approaches to estimate rare-event probabilities, propagate uncertainty via Bayesian quadrature, or efficiently compute global sensitivity indices (e.g., Sobol’ indices), leveraging the full posterior (Li et al., 5 Feb 2025).

5. Extensions: Hierarchical, Deep, and Physics-informed GPs

Deep Gaussian Processes (DGPs) generalize classical GPs to multilayer hierarchical models, composing GP mappings to model complex, nonlinear relationships. Uncertainty propagates layer by layer, captured by variational inference over inducing points; predictive distributions retain Bayesian calibration. Deep Sigma-Point Processes (DSPPs) introduce deterministic quadrature for propagating uncertainty through layers, producing Gaussian mixture outputs with empirical advantages in calibration (Lende et al., 24 Apr 2025, Daneshkhah et al., 2020).

Hybrid GP frameworks incorporate physics by embedding variational or residual loss terms into the training objective (e.g., Boltzmann-Gibbs distribution regularization based on PDE residuals),

$$\mathcal{L}(\theta) = \beta L^*(y, X) + y^T[K_\theta(X, X) + \sigma^2 I]^{-1} y + \log \det[K_\theta(X, X) + \sigma^2 I]$$

This joint data-driven and physics-constrained approach enables reliable UQ when labeled data is scarce (Chang et al., 2022).

Operator learning architectures embed neural operators into GP kernels, constructing latent-space kernels whose input resolution independence allows accurate and scalable uncertainty-aware learning for parametric PDEs, facilitated by stochastic dual descent optimization (Kumar et al., 2024).

6. Specialized Algorithms and Applications

Inverse Uncertainty Quantification and Discrepancy Modeling

Modular Bayesian inverse UQ frameworks deploy GPs both as surrogate emulators for expensive simulation codes and to model systematic discrepancy functions between simulations and physical observations. This two-stage GP modeling avoids overfitting, delivers calibrated posteriors for physical parameters, and enables MCMC sampling with orders-of-magnitude speedups. Sequential test-source allocation (TSA) further maximizes the informativeness of available data (Wu et al., 2018).

Conformal and Martingale-based Uncertainty Quantification

Conformalized GP methods integrate online conformal prediction to guarantee valid coverage even under model misspecification, particularly for streaming graph-structured data. Ensemble RF-GP models achieve adaptive coverage and interval width with near-nominal empirical guarantees (Xu et al., 7 Oct 2025). Martingale-based confidence sequences for GPs yield frequentist confidence bands robust to prior mis-specification, ensuring valid coverage for Bayesian optimization applications (Neiswanger et al., 2020).

Domain-specific Kernels and Structural Inputs

GAUCHE provides domain-specific kernels for chemical graphs, SMILES strings, and molecular fingerprints, enabling calibrated UQ in chemistry applications. GPs on structured discrete spaces—graphs, strings, bit vectors—support Bayesian optimization and reliable regression in ultra-low-data regimes, with closed-form posterior variance-calibrated uncertainty (Griffiths et al., 2022).

Activation-level Post-hoc UQ in Neural Networks

Post-hoc GP activation functions (GAPA) model uncertainty at neuron activations rather than in weight space, preserving original network mean predictions. Both empirical and variational versions yield efficient uncertainty propagation in large-scale neural networks, outperforming Laplace approximations in calibration metrics (Bergna et al., 28 Feb 2025).

7. Practical Implications and Limitations

Gaussian process uncertainty quantification frameworks achieve:

• Principled calibration of predictive distributions, with empirical coverage matching nominal values in many domains.
• Reliable propagation of both aleatoric and epistemic uncertainty, separated via analytic or hierarchical GP constructions.
• Dramatic reductions in computational cost for UQ over conventional Monte Carlo via nonparametric emulation and dimensionality reduction.
• Robust decision making under uncertainty in optimization, reliability, and sensitivity tasks, with theoretical guarantees when adaptive penalization and acquisition functions are used.

Limitations include cubic scaling for exact GPs (requiring sparse or random-feature approximations), sensitivity of standard posterior intervals to prior mis-specification unless martingale or conformal mechanisms are used, and empirical degradation in hierarchical models under covariate shift relative to deep ensembles (Lende et al., 24 Apr 2025).

A plausible implication is that the deployment of GP-based UQ tools should include empirical stress tests under covariate shifts, misspecified priors, and realistic data scarcity to ensure robustness. Physics-informed and operator-latent GP approaches offer systematic mechanisms to leverage domain knowledge, while activation-level GP surrogates and conformal adaptations extend uncertainty quantification to modern neural architectures and graph-structured domains.

---

Key Papers Referenced: (Li et al., 5 Feb 2025, Hu et al., 2019, Bilionis et al., 2016, Jung et al., 7 Nov 2025, Gadde, 20 Jul 2025, Daneshkhah et al., 2020, Chang et al., 2022, Kumar et al., 2024, Lende et al., 24 Apr 2025, Xu et al., 7 Oct 2025, Griffiths et al., 2022, Wu et al., 2018, Bergna et al., 28 Feb 2025, Ajirak et al., 7 Sep 2025, Neiswanger et al., 2020, Ju, 4 Dec 2025, Hoche et al., 16 Dec 2025, Dance et al., 2024, Azari et al., 2024, Feng et al., 30 Jul 2025).