Physics-Informed Gaussian Process Regression

• PI-GPR is a modeling approach that integrates physical laws as operator constraints into GP priors, ensuring physically consistent predictions.
• The method combines simulation-based PhIK with high-fidelity data using CoPhIK to reduce hyperparameter tuning and enhance prediction accuracy.
• Active learning and uncertainty quantification are leveraged to optimize sensor placement and surrogate performance in complex systems.

Physics-Informed Gaussian Process Regression (PI-GPR) integrates mechanistic knowledge—such as physical laws expressed by differential or algebraic operators—directly into the Gaussian process (GP) regression framework. This approach systematically fuses stochastic physics-based models and observational data, constructing surrogates that are both uncertainty-aware and physically consistent. The resulting methodology underpins a spectrum of modeling strategies, including simulation-aided Kriging, operator-constrained GPs, multi-fidelity fusion, and hybrid data–physics kernel learning.

1. Fundamentals of PI-GPR and Operator-Constrained Gaussian Processes

In the standard GP regression paradigm, a prior is assumed for an unknown function $u(x)$ as $u \sim \mathrm{GP}(m(x), k(x, x'))$, where $k$ is typically a stationary kernel with hyperparameters optimized from data. PI-GPR departs from this by constructing priors such that the realizations or the mean/covariance inherit constraints imposed by physical operators (e.g., differential, integral, or algebraic constraints).

For linear operator constraints $\mathcal{L}$ (e.g., $\mathcal{L}[u] = f$ for PDEs), a PI-GPR prior can be built such that:

• The mean and covariance functions, possibly non-stationary, are computed empirically from realizations of a stochastic physics model, e.g., by Monte Carlo or multilevel Monte Carlo (MLMC) (Yang et al., 2018).
• The constraint $\mathcal{L}[u]=f$ is approximately enforced in the prior, so the posterior mean satisfies the physical law up to a rigorously bounded residual.
• No kernel hyperparameter optimization is required when the covariance is estimated directly from stochastic simulations ("physics-informed Kriging", PhIK).

For instance, letting $u(x;\omega)$ denote a stochastic PDE solution with $M$ realizations, the empirical mean and covariance are: $$m_L(x) = \frac{1}{M}\sum_{m=1}^M Y^m(x), \quad k_L(x, x') = \mathrm{Cov}_\text{MC}[Y(x), Y(x')] \ .$$ By linearity, the operator acts on the prior so that the mean and covariance satisfy: $$\mathcal{L}m_L(x) \approx f(x), \qquad \mathcal{L}_x k_L(x, x') \approx \mathrm{Cov}(f(x), Y(x')) \ .$$ This builds a non-stationary, data-driven prior imbued with approximate physics (Yang et al., 2018).

2. Multi-Fidelity and Physics-Informed CoKriging (CoPhIK)

High-fidelity experimental data and lower-fidelity simulation surrogates can be fused using CoKriging, forming a two-level GPR:

• The low-fidelity GP $u \sim \mathrm{GP}(m(x), k(x, x'))$0 is constructed via PhIK.
• The high-fidelity prediction is modeled as $u \sim \mathrm{GP}(m(x), k(x, x'))$1, where $u \sim \mathrm{GP}(m(x), k(x, x'))$2 is a stationary "discrepancy GP" with its own kernel hyperparameters.

Hyperparameters for the discrepancy GP (variance, lengthscales), the regression coefficient $u \sim \mathrm{GP}(m(x), k(x, x'))$3, and possible offset are learned via log-marginal likelihood of residuals between high-fidelity data and the upscaled low-fidelity prediction. As the low-fidelity covariance is physics-driven and non-parametric, the parameter search space is substantially reduced (Yang et al., 2018).

Block-covariance structure for the joint $u \sim \mathrm{GP}(m(x), k(x, x'))$4: $u \sim \mathrm{GP}(m(x), k(x, x'))$5 Prediction at a test location and associated variance have explicit formulas, and the influence of physical constraint enforcement can be rigorously bounded (see Section 5).

3. Predictive Formulas, Hyperparameter Estimation, and Error Bounds

For the bi-fidelity model, the high-fidelity posterior mean and variance at $u \sim \mathrm{GP}(m(x), k(x, x'))$6 are computed via standard block-GP equations: $u \sim \mathrm{GP}(m(x), k(x, x'))$7 with $u \sim \mathrm{GP}(m(x), k(x, x'))$8 and vector $u \sim \mathrm{GP}(m(x), k(x, x'))$9 derived as above.

Hyperparameter learning in CoPhIK is restricted to the discrepancy kernel and coupling coefficient, minimizing the log-marginal likelihood of the residuals. The crucial distinction is that the low-fidelity kernel is informed by physics and requires no fitting.

Physical constraint satisfaction can be guaranteed up to a computable error bound: for realizations satisfying $k$0, the CoPhIK predictor $k$1 satisfies

$k$2

where $k$3 are terms (explicit in (Yang et al., 2018)) capturing Monte Carlo sampling and discrepancy GP error.

4. Active Learning and Uncertainty-Based Experiment Design

PI-GPR and CoPhIK naturally support active learning schemes. At each iteration, the model identifies the location with maximal posterior predictive variance (mean-squared prediction error), selects it for measurement, and updates the surrogate. This process focuses sampling where model uncertainty remains largest: $k$4 Active learning with CoPhIK and PhIK robustly outperforms purely data-driven Kriging in terms of speed and ultimate accuracy in reconstructing complex fields from sparse measurements (Yang et al., 2018, Yang et al., 2018).

5. Numerical Performance and Applications

Extensive numerical experiments confirm the superiority of PI-GPR frameworks:

• Modified Branin function: With eight initial points, standard Kriging incurs >50% error, PhIK achieves ~8%, and CoPhIK <3%. Under active learning (24 total points), CoPhIK error drops below 0.1%—over an order of magnitude better than any baselines.
• Steady heat conduction with nonlinear conductivity: With six boundary sensors, Kriging yields ~27% error, PhIK ~7.8%, CoPhIK ~4.8%. Using 22 points with active learning, CoPhIK error falls below 1%.
• Conservative tracer transport in random media: With combined MLMC for PhIK, Kriging error is ~50%, PhIK ~20%, CoPhIK ~17%. As the stochastic model approaches reality, PhIK slightly outperforms CoPhIK, both vastly exceeding the accuracy of standard Kriging.

Practical applications span surrogate modeling of expensive PDE systems, Bayesian calibration, sensor placement, and data-driven correction of simulation bias, particularly for problems where the physical model is informative but imperfect.

6. Comparison to Alternative Physics-Informed GP Strategies

• Standard PI-GPR employs kernel construction or hyperparameter tuning such that sample paths or covariance structures satisfy physical operator constraints exactly or in mean. These methods require high-dimensional, potentially nonconvex kernel parameter optimization and can become computationally prohibitive for complex or nonlinear operators.
• Pure PhIK produces a physics-consistent nonstationary GP prior but may inherit systematic bias from the underlying stochastic model, converging more slowly to observations if the model is misspecified.
• Standard CoKriging fuses high- and low-fidelity data absent any explicit encoding of physical structure in the low-fidelity GP.
• CoPhIK merges the strengths of both: physics-informed non-parametric prior, minimal hyperparameter search, and rapid data-driven adaptation. It is robust for moderately reliable physical models and stationary discrepancy structures. Limitations arise when model bias is substantial or the discrepancy is highly non-stationary—suggesting the need for more general discrepancy kernels or nonlinear multi-fidelity architectures.

7. Limitations and Scope of Applicability

CoPhIK and related PI-GPR multi-fidelity surrogates hinge on the fidelity and representativeness of the underlying stochastic simulation model used for prior construction. Severe model discrepancies or non-stationary residuals between the simulation and reality may challenge the surrogate's validity, requiring either more flexible GP structures or more sophisticated coupling schemes. For high-dimensional parameter spaces or computationally expensive simulations, variance-reduction via MLMC and judicious active learning are critical for feasibility (Yang et al., 2018, Yang et al., 2018).

Applications include:

• Data-efficient surrogate modeling, enabling uncertainty quantification and optimization in computational physics,
• Sensor placement and experimental design driven by GP uncertainty estimates,
• Hybrid calibration in environmental, climate, and engineering systems where high-fidelity observations are sparse and simulation is available but imperfect.

---

PI-GPR—exemplified by CoPhIK—systematically integrates physical knowledge into the GP regression framework, yielding uncertainty-aware, constraint-respecting surrogates that efficiently combine stochastic simulation models and data (Yang et al., 2018, Yang et al., 2018).