Physics-Informed Log Evidence (PILE)

• PILE is an evidential framework that unifies data and physics constraints into a single scalar objective based on normalized negative log marginal likelihood.
• It leverages Gaussian process models and augmented covariances to simultaneously measure data misfit and epistemic uncertainty.
• Minimizing PILE aids in calibrating kernel selection and regularization weights, thereby diagnosing overfitting and guiding robust model improvements.

Searching arXiv for the cited papers and closely related context. Searching arXiv for "Uncertainty-Aware Diagnostics for Physics-Informed Machine Learning" and "Physics Informed Deep Kernel Learning". Physics-Informed Log Evidence (PILE) is an uncertainty-aware evidential criterion for physics-informed machine learning that replaces separate monitoring of data loss and physics residual loss by a single scalar objective grounded in marginal likelihood. In the Gaussian-process formulation developed in "Uncertainty-Aware Diagnostics for Physics-Informed Machine Learning" (Daniels et al., 30 Oct 2025), PILE is the normalized negative log marginal likelihood of an augmented observation model containing both measured data and physics-residual pseudo-observations. In the deep-kernel setting of "Physics Informed Deep Kernel Learning" (Wang et al., 2020), the same evidential idea appears as a collapsed physics-informed log-evidence term inside an ELBO, where the differential-operator residual is scored under a Gaussian-process prior on latent sources. Across these formulations, the central aim is the same: to make model selection and diagnostics uncertainty-aware in multi-objective physics-informed learning.

1. Conceptual role in physics-informed learning

Physics-informed machine learning (PIML) integrates prior physical information, often in the form of differential equation constraints, into the process of fitting machine learning models to physical data. Popular PIML approaches, including neural operators, physics-informed neural networks, neural ordinary differential equations, and neural discrete equilibria, are typically fit to objectives that simultaneously include both data and physical constraints. The multi-objective nature of this approach creates ambiguity in the measurement of model quality, and this is related to a poor understanding of epistemic uncertainty (Daniels et al., 30 Oct 2025).

PILE addresses that ambiguity by elevating the marginal likelihood, or Bayesian evidence, to the setting where both data misfit and physics-residual penalties appear. In the Gaussian-process setting, it is explicitly described as an uncertainty-aware, single-number diagnostic and model selection criterion. Its purpose is not merely to aggregate losses, but to unify fit and epistemic uncertainty probabilistically. The quadratic form measures fit under the model covariance, while the log-determinant term penalizes posterior uncertainty and effective model complexity. Lower PILE values are preferred (Daniels et al., 30 Oct 2025).

This evidential perspective distinguishes PILE from validation losses and residual norms. Validation or test losses isolate fit but ignore epistemic uncertainty and its coupling to physics; residual norms of differential operators measure constraint violation but miss calibration and the trade-off with data fidelity. A plausible implication is that PILE is most useful precisely when apparently good residual scores conceal unstable or weakly identified solutions.

2. Gaussian-process formulation and exact definition

The formal construction in (Daniels et al., 30 Oct 2025) begins with a latent function $f$ on a domain $\Omega \subset \mathbb{R}^d$, endowed with a Gaussian-process prior

$f \sim \mathcal{GP}(m, k_\theta),$

where $m : \Omega \to \mathbb{R}$ is a mean function and $k_\theta : \Omega \times \Omega \to \mathbb{R}$ is a positive-definite kernel with hyperparameters $\theta$. Under noisy observations $y_i$ at input locations $x_i \in \Omega$,

$$y_i = f(x_i) + \epsilon_i, \qquad \epsilon_i \sim \mathcal{N}(0,\sigma^2), \quad i=1,\dots,n.$$

With $X=(x_i)_{i=1}^n$, $\Omega \subset \mathbb{R}^d$0, and Gram matrix $\Omega \subset \mathbb{R}^d$1, the standard GP log marginal likelihood is

$\Omega \subset \mathbb{R}^d$2

This already trades off data fit and posterior uncertainty.

Physics enters through linear operators. The interior PDE operator is denoted $\Omega \subset \mathbb{R}^d$3 or $\Omega \subset \mathbb{R}^d$4, with examples such as $\Omega \subset \mathbb{R}^d$5 for a Poisson residual and $\Omega \subset \mathbb{R}^d$6 for a convection residual. Boundary operators $\Omega \subset \mathbb{R}^d$7 may encode Dirichlet, Neumann, or Robin conditions. Rather than imposing these constraints as hard equalities, the framework treats them as noisy pseudo-observations with Gaussian noise, collected at quadrature points $\Omega \subset \mathbb{R}^d$8 with quadrature weights $\Omega \subset \mathbb{R}^d$9 (Daniels et al., 30 Oct 2025).

Let $f \sim \mathcal{GP}(m, k_\theta),$0 denote the stacked multi-operator containing the interior and boundary operators. Define

$f \sim \mathcal{GP}(m, k_\theta),$1

$f \sim \mathcal{GP}(m, k_\theta),$2

$f \sim \mathcal{GP}(m, k_\theta),$3

With prior scale $f \sim \mathcal{GP}(m, k_\theta),$4 and observation-noise scales $f \sim \mathcal{GP}(m, k_\theta),$5 for data and $f \sim \mathcal{GP}(m, k_\theta),$6 for physics, the augmented covariance is

$f \sim \mathcal{GP}(m, k_\theta),$7

If $f \sim \mathcal{GP}(m, k_\theta),$8 denotes the concatenation of observed data and physics residual pseudo-observations, then the Physics-Informed Log Evidence is

$f \sim \mathcal{GP}(m, k_\theta),$9

This exact form is the central object of the 2025 framework (Daniels et al., 30 Oct 2025).

The paper also gives an equivalent generalized form for separate interior and boundary residual blocks:

$m : \Omega \to \mathbb{R}$0

with $m : \Omega \to \mathbb{R}$1. Either representation is a free-energy-based scalar whose minimization selects kernel parameters and regularization weights in an uncertainty-aware manner.

3. Data-free PILE and the Fredholm-determinant limit

A distinctive feature of the framework in (Daniels et al., 30 Oct 2025) is the data-free case, obtained by setting $m : \Omega \to \mathbb{R}$2 and enforcing physics solely through pseudo-observations at quadrature points. In that case,

$m : \Omega \to \mathbb{R}$3

When $m : \Omega \to \mathbb{R}$4, corresponding to homogeneous constraints, the quadratic term vanishes and the score reduces to a normalized log-determinant.

The asymptotic statement proved in Section 4 of (Daniels et al., 30 Oct 2025) identifies the limiting object. Let

$m : \Omega \to \mathbb{R}$5

acting on $m : \Omega \to \mathbb{R}$6, and define

$m : \Omega \to \mathbb{R}$7

Then

$m : \Omega \to \mathbb{R}$8

Thus, the data-free PILE converges, up to known normalizations, to the Fredholm determinant of an integral operator determined by the PDE operator $m : \Omega \to \mathbb{R}$9 and the kernel $k_\theta : \Omega \times \Omega \to \mathbb{R}$0 (Daniels et al., 30 Oct 2025).

This result gives PILE a prior-only role: it can identify kernels that are "well-adapted" to a given PDE before any data are acquired. The paper further suggests the calibration choice

$k_\theta : \Omega \times \Omega \to \mathbb{R}$1

to align normalized and unnormalized variants. This suggests that kernel selection can be split into an a priori stage, driven by operator-kernel compatibility, and an a posteriori stage, driven by joint data-and-physics evidence.

4. Optimization, gradients, and computation

PILE minimization is proposed for selecting kernel parameters $k_\theta : \Omega \times \Omega \to \mathbb{R}$2, regularization and noise weights $k_\theta : \Omega \times \Omega \to \mathbb{R}$3, and even operator weighting through $k_\theta : \Omega \times \Omega \to \mathbb{R}$4. The paper reports that PILE minimization yields excellent choices for a wide variety of model parameters, including kernel bandwidth, least squares regularization weights, and kernel function selection (Daniels et al., 30 Oct 2025).

For the normalized objective

$k_\theta : \Omega \times \Omega \to \mathbb{R}$5

the derivative with respect to any scalar parameter $k_\theta : \Omega \times \Omega \to \mathbb{R}$6 is

$k_\theta : \Omega \times \Omega \to \mathbb{R}$7

The relevant identities are

$k_\theta : \Omega \times \Omega \to \mathbb{R}$8

Examples of covariance derivatives are given explicitly for $k_\theta : \Omega \times \Omega \to \mathbb{R}$9, $\theta$0, $\theta$1, and $\theta$2 in block form (Daniels et al., 30 Oct 2025).

Efficient evaluation uses Cholesky factorization $\theta$3. The quadratic term is computed as $\theta$4, the log determinant as $\theta$5, and trace terms can be estimated by Hutchinson’s estimator for large $\theta$6. Dense Cholesky has $\theta$7 time and $\theta$8 memory complexity. The paper recommends GPU-enabled libraries such as GPyTorch, determinant approximations under memory constraints, block-structure exploitation through Schur complements and Woodbury identities, and numerical stabilization through jitter $\theta$9 with $y_i$0–$y_i$1 times $y_i$2 (Daniels et al., 30 Oct 2025).

The practical implementation sequence is: choose kernel $y_i$3 and quadrature points $y_i$4 with weights $y_i$5; assemble $y_i$6, $y_i$7, and $y_i$8; form $y_i$9; compute the Cholesky factor; evaluate the quadratic and log-determinant terms; and minimize over $x_i \in \Omega$0. Type-1 Chebyshev grids are used in the paper as an example of quadrature with fast convergence and good $x_i \in \Omega$1 integration properties.

5. Evidential variants in GP PILE and PI-DKL

The term "Physics-Informed Log Evidence" spans at least two closely related evidential constructions in the supplied literature. The 2025 diagnostic paper defines an exact augmented log evidence in a Gaussian-process regression model with explicit data and physics blocks (Daniels et al., 30 Oct 2025). The 2020 PI-DKL formulation places the same evidential principle inside a deep-kernel Bayesian objective, where the differential-operator residual is evaluated under a latent-source GP prior and contributes an additive expected log-likelihood term to a collapsed ELBO (Wang et al., 2020).

Setting	Objective form	Role of physics
GP diagnostics	$x_i \in \Omega$2 from augmented covariance $x_i \in \Omega$3	Pseudo-observations of residuals
PI-DKL	$x_i \in \Omega$4	Expected log-likelihood of residual under source prior

In PI-DKL, the supervised model uses a deep-kernel GP prior

$x_i \in \Omega$5

with Gaussian observation model

$x_i \in \Omega$6

Physics knowledge is expressed as a differential equation $x_i \in \Omega$7, where $x_i \in \Omega$8 is an unknown latent source with GP prior. For random physics locations $x_i \in \Omega$9, the model forms a posterior sample surrogate

$$y_i = f(x_i) + \epsilon_i, \qquad \epsilon_i \sim \mathcal{N}(0,\sigma^2), \quad i=1,\dots,n.$$0

then applies the operator to obtain

$$y_i = f(x_i) + \epsilon_i, \qquad \epsilon_i \sim \mathcal{N}(0,\sigma^2), \quad i=1,\dots,n.$$1

After marginalizing the latent source $$y_i = f(x_i) + \epsilon_i, \qquad \epsilon_i \sim \mathcal{N}(0,\sigma^2), \quad i=1,\dots,n.$$2, the log evidence becomes

$$y_i = f(x_i) + \epsilon_i, \qquad \epsilon_i \sim \mathcal{N}(0,\sigma^2), \quad i=1,\dots,n.$$3

and the collapsed ELBO is

$$y_i = f(x_i) + \epsilon_i, \qquad \epsilon_i \sim \mathcal{N}(0,\sigma^2), \quad i=1,\dots,n.$$4

The physics term therefore acts as posterior regularization, penalizing the expected squared residual of the differential equation in the precision metric $$y_i = f(x_i) + \epsilon_i, \qquad \epsilon_i \sim \mathcal{N}(0,\sigma^2), \quad i=1,\dots,n.$$5 (Wang et al., 2020).

For linear operators $$y_i = f(x_i) + \epsilon_i, \qquad \epsilon_i \sim \mathcal{N}(0,\sigma^2), \quad i=1,\dots,n.$$6, PI-DKL gives the explicit decomposition

$$y_i = f(x_i) + \epsilon_i, \qquad \epsilon_i \sim \mathcal{N}(0,\sigma^2), \quad i=1,\dots,n.$$7

so that the expected quadratic form splits into a residual of the posterior mean and a residual of the uncertainty shape. This makes the relation to uncertainty-aware diagnostics direct: physics constrains both the central prediction and the posterior spread.

6. Empirical behavior, comparisons, and limitations

The case studies in (Daniels et al., 30 Oct 2025) are organized around Poisson and convection problems. For a 2D Poisson equation with Dirichlet boundary conditions, PILE minimized over kernel bandwidth $$y_i = f(x_i) + \epsilon_i, \qquad \epsilon_i \sim \mathcal{N}(0,\sigma^2), \quad i=1,\dots,n.$$8 and weights $$y_i = f(x_i) + \epsilon_i, \qquad \epsilon_i \sim \mathcal{N}(0,\sigma^2), \quad i=1,\dots,n.$$9 selects $X=(x_i)_{i=1}^n$0 so as to avoid undersmoothing and oversmoothing while balancing data and physics generalization errors. The paper reports that PILE diverges when $X=(x_i)_{i=1}^n$1 or $X=(x_i)_{i=1}^n$2 are too small, and interprets this as a correct diagnosis of overfitting. In a convection PDE example, an isotropic RBF kernel exhibits no bandwidth that fits both data and physics; PILE diagnoses this by preferring an "all-zero" oversmoothed solution that is safe but uninformative. Expanding the kernel family to anisotropic RBFs changes the evidential landscape, and the data-free PILE identifies a kernel prior to data acquisition; after that adjustment, PILE minimization yields excellent joint fits (Daniels et al., 30 Oct 2025).

These behaviors motivate the comparison with conventional alternatives. Standard GP evidence optimizes against observed data only, whereas PILE augments the evidence with physics pseudo-observations. Validation losses and residual norms do not include a log-determinant penalty for epistemic uncertainty. PINNs minimize squared residual norms at collocation points without a probabilistic marginal likelihood; LFMs hardwire physics via convolution and require linear operators and Green’s functions. In PI-DKL, the evidential physics term is described as a principled expected log-likelihood of the physics residual under the source prior, and reported results show improved RMSE and test log-likelihood over SKL and DKL in extrapolation and data-scarce regimes, with task-specific values including 1stODE RMSEs $X=(x_i)_{i=1}^n$3 for SKL, DKL, LFM, and PI-DKL, respectively, and 1dDiffusion RMSEs $X=(x_i)_{i=1}^n$4 (Wang et al., 2020).

The main limitations are numerical and modeling-related. Operator-applied covariances such as $X=(x_i)_{i=1}^n$5 can be ill-conditioned, especially for high-order differential operators or clustered quadrature points; suggested remedies include jitter, input scaling, normalization of $X=(x_i)_{i=1}^n$6, improved quadrature design, diagonal scaling, and Schur-complement-based solves (Daniels et al., 30 Oct 2025). If $X=(x_i)_{i=1}^n$7 or $X=(x_i)_{i=1}^n$8 are misspecified, PILE may favor degenerate or overly smooth solutions, so they should be treated as tunable hyperparameters and may be placed in a hierarchical Bayesian setting. Inadequate boundary sampling or inaccurate Hausdorff measure weights can distort the score. In broader neural settings, open challenges include non-Gaussianity, multimodality, strong nonlinear physics, stable operator-applied NTK blocks, and determinant scaling (Daniels et al., 30 Oct 2025).

Taken together, these formulations present PILE as an evidence-based response to the multi-objective ambiguity of physics-informed learning. In the kernel GP setting it is an exact normalized negative log marginal likelihood on an augmented covariance; in PI-DKL it is a collapsed expected log-likelihood term regularizing the posterior through the differential operator. In both cases, the governing idea is the same: model selection should be based not only on how well data and residuals are fitted, but also on how much epistemic uncertainty remains after physics has been taken into account (Daniels et al., 30 Oct 2025).