Physics-Informed Kernel Learning

• Physics-Informed Kernel Learning is a framework that integrates kernel methods with explicit physical laws to solve supervised and inverse problems under PDE constraints.
• It employs modified RKHS norms, Fourier/spectral techniques, and Green’s functions to embed differential operators directly into the kernel structure.
• The approach offers robust statistical guarantees, enhanced scalability, and improved uncertainty quantification across applications like PDE solvers and system identification.

Physics-Informed Kernel Learning (PIKL) refers to a rigorous class of machine learning methodologies in which kernel-based methods are systematically augmented with explicit knowledge of governing physical laws—typically encoded as differential, difference, or integral operators—yielding models that enjoy both the expressiveness of data-driven learning and the inductive structure of physical constraints. PIKL provides frameworks for regression, system identification, generative modeling under physical laws, and uncertainty quantification, with provable statistical properties and scalable implementations. This entry surveys the theoretical foundations, algorithmic realizations, statistical performance, computational aspects, and representative applications of PIKL.

1. Mathematical Foundations and Problem Formulation

PIKL formalizes supervised and inverse problems through the minimization of a physics-informed empirical risk, incorporating three terms: data fidelity, smoothness regularization, and a penalty or constraint derived from known physical laws, often modeled by linear PDE operators. The canonical form is: $$R_n(f) = \frac{1}{n}\sum_{i=1}^n (f(x_i) - y_i)^2 + \lambda \|f\|^2_{\mathcal{H}} + \mu \|\mathcal{D} f\|^2_{L^2(\Omega)},$$ where $\mathcal{D}$ is a differential operator representing the physics (e.g., Laplacian, heat, or wave operator), $\mathcal{H}$ is an RKHS, and $\lambda,\mu$ regulate the strength of the smoothness and physical constraints (Doumèche et al., 2024, Doumèche et al., 2024, Doumèche et al., 2 Sep 2025). For nonlinear or hybrid models, similar objectives are constructed, blending a parametric physical model with a nonparametric kernel correction (Donati et al., 9 Sep 2025, Thorpe et al., 2023).

By the generalized representer theorem, the minimizer $\hat{f}$ always lies in the span of kernel functions evaluated at data locations, and, crucially, the regularization norm typically becomes

$$\|f\|^2_{\mathcal{H}} + \mu \|\mathcal{D} f\|^2_{L^2}$$

which induces a new, physics-informed kernel $K_\mu$ whose construction involves both the native RKHS norm and the physical operator (Doumèche, 11 Jul 2025, Doumèche et al., 2024, Doumèche et al., 2 Sep 2025). In the common case $\mathcal{H} = H^s(\Omega)$ (Sobolev space), these constraints are tractable via Fourier or eigenfunction expansions.

2. Physics-Informed Kernel Construction and Implementation

Physics-informing a kernel can proceed via explicit modification of the RKHS inner product, spectral (Fourier) characterization, or direct embedding of known physical solutions (e.g., Green's functions, Trefftz bases).

• Fourier/Spectral Approach: For constant-coefficient operators and periodic or simple domains, expansion in a Fourier (e.g., trigonometric) basis allows the norm

$$\|f\|^2_{H^s} + \mu \sum_{|k|\leq m} |p(k)|^2 |\langle f, \phi_k \rangle|^2$$

yielding physics-informed kernels

$$K_\mu(x,y) = \sum_{k} \frac{\phi_k(x)\overline{\phi_k(y)}}{(1+\|k\|^2)^s + \mu |p(k)|^2}$$

where $\mathcal{D}$0 is the symbol of $\mathcal{D}$1 (Doumèche et al., 2024, Doumèche, 11 Jul 2025). This approach enables scalable $\mathcal{D}$2 solvers via NUFFT (Doumèche et al., 2 Sep 2025).

• Fundamental Solution Initiated Kernels: For certain operators, the hidden layer of a neural network may be populated by PDE-specific kernels—e.g., the heat, wave, or Helmholtz Green's function—instead of generic RBFs, directly encoding physical constraints into the architecture (Fu et al., 2023, Jiao et al., 2024).
• Hybrid and Data-Driven Kernels: In hybrid system identification, the baseline model is the sum of a parametric physics-derived component and a nonparametric kernel component, with the kernel itself potentially parameterized by a neural network for deep kernels (Donati et al., 9 Sep 2025, Yan et al., 30 Jan 2025, Wang et al., 2020).
• Operator-Driven Kernels in Generative Models: PIKL has been generalized to generative architectures, where the "kernel" governs the evolution of a probability distribution under a continuity (Wegner) equation, preserving probability weight exactly—a property exploited to solve sign problems in quantum field theory (Ihssen et al., 3 Mar 2026).

3. Algorithmic Procedures and Uncertainty Quantification

Algorithmic implementation in PIKL typically involves:

1. Model Selection: Choose basis (Fourier, Chebyshev, Green's) and construct physics-informed kernel, possibly with BO or validation-based tuning of hyperparameters (Dwivedi et al., 14 Jul 2025, Daniels et al., 30 Oct 2025).
2. Linear System Solve: For fixed kernel (or kernel parameters), fit model coefficients in dual (kernel) or primal (basis) form. For large-scale problems, employing iterative solvers (e.g., conjugate gradient) with FFT acceleration is standard (Doumèche et al., 2 Sep 2025, Doumèche et al., 2024).
3. Hyperparameter Optimization: Recent advances use the Physics-Informed Log Evidence (PILE)—a normalized GP marginal likelihood depending jointly on data and PDE residuals—to select kernel families, bandwidths, and all regularization weights in a fully Bayesian framework (Daniels et al., 30 Oct 2025). The minimized PILE is empirically correlated with generalization and residual error.
4. Uncertainty Quantification: GP-based PIKL methods provide principled posterior mean and covariance for predictions, with closed-form expressions that propagate physical constraints (Yan et al., 30 Jan 2025, Daniels et al., 30 Oct 2025). Variational approaches optimize an evidence lower bound (ELBO) that acts as a physics-aware posterior regularizer (Wang et al., 2020).

4. Generalization Theory, Statistical Guarantees, and Overfitting

PIKL enjoys rigorous statistical guarantees that extend classical kernel learning results:

• Minimax-Optimality: When the physical law (e.g., $\mathcal{D}$3) holds exactly, convergence of the PIKL estimator can improve from the Sobolev minimax rate $\mathcal{D}$4 to a nearly parametric rate $\mathcal{D}$5 (Doumèche et al., 2024, Doumèche et al., 2024, Doumèche et al., 2 Sep 2025, Doumèche, 11 Jul 2025).
• Impact of Model Misspecification: When $\mathcal{D}$6 only approximately satisfies the physics, the excess risk decomposes into a physics penalty and data term, with explicit dependence on the PDE-misfit $\mathcal{D}$7 (Doumèche et al., 2024).
• Benign Overfitting and Spectral Stabilization: For PDE-constrained inverse problems, the operator $\mathcal{D}$8 can act as a spectral stabilizer—variance remains bounded even in interpolation regimes and with ridgeless estimators (benign overfitting). The convergence rate becomes independent of Sobolev order once smoothness exceeds a Bayesian threshold matching results in classical nonparametric Bayesian theory (Wong et al., 2024).
• Effective Dimension and Data-Physics Trade-Off: Rates and complexity of PIKL estimators are characterized by the effective dimension of the physics-informed kernel, directly controlled by regularization parameters. Hyperparameter choice is crucial to balance data fidelity, smoothness, and physics enforcement (Doumèche et al., 2024, Doumèche, 11 Jul 2025).

5. Applications and Empirical Performance

PIKL methods have been demonstrated across a variety of scientific, engineering, and control tasks, including:

• High-Dimensional PDE Solvers: PDE-constrained deep kernel learning (PDE-DKL) has shown dramatic accuracy and tractability gains over plain GPs or PINNs for high-dimensional parametric PDEs (e.g., up to 50 spatial dimensions; accuracy $\mathcal{D}$9, $\mathcal{H}$0) (Yan et al., 30 Jan 2025).
• System Identification and Control: Hybrid grey-box modeling—parametric physics plus nonparametric kernel correction—yields state-of-the-art accuracy in nonlinear dynamical-system identification (cascade-tank benchmarks, $\mathcal{H}$1-step RMSE $\mathcal{H}$2, simulation fit $\mathcal{H}$3) (Donati et al., 9 Sep 2025). Closed-form kernel mean embeddings further integrate approximate physics into data-driven control with enhanced sample efficiency (Thorpe et al., 2023).
• Physical Reasoning and Causal Search: Causal-PIK exploits a dynamics-predictor-informed kernel to accelerate active search in physical-reasoning benchmarks, outperforming state-of-the-art and even matching human performance on the PHYRE and Virtual Tools tasks (Parés-Morlans et al., 28 May 2025).
• Green's Function and Operator Learning: Neural networks parametrized by fundamental solutions or Green's kernels (GsPINN, PIKFNN) enable efficient and accurate computation of PDE fundamental solutions, with superior convergence on high-wavenumber and infinite-domain problems (Jiao et al., 2024, Fu et al., 2023).
• Uncertainty-Aware Model Selection: The PILE score enables uncertainty-informed hyperparameter optimization and kernel family selection, automating model selection for physics-constrained GPs and extending to data-free kernel adaptation prior to data acquisition (Daniels et al., 30 Oct 2025).

6. Variants, Extensions, and Practical Guidelines

• Neural and Deep Kernel Extensions: Physics-informed deep kernel learning combines neural networks for latent representation with GP architectures subject to explicit PDE constraints, delivering high accuracy and robust uncertainty quantification in data-scarce, high-dimensional settings (Yan et al., 30 Jan 2025, Wang et al., 2020).
• Spectral and Domain Decomposition: Kernel spectral analysis via the neural tangent kernel (NTK) theory has connected convergence rates and training dynamics to the eigenvalue structure of physics-informed kernels. Domain decomposition strategies, normalization, and orthogonal basis selection (e.g., Chebyshev, Fourier) are prescribed to optimally condition the NTK and accelerate convergence—especially in stiff or multiscale problems (Faroughi et al., 9 Jun 2025).
• Scaling and Large-Scale Solvers: Fourier-based physics-informed kernel computation with NUFFT enables handling extremely large datasets (up to $\mathcal{H}$4) in applications such as time series forecasting, with guaranteed statistical and computational efficiency (Doumèche et al., 2 Sep 2025, Doumèche, 11 Jul 2025).
• Inverse and Sign-Problem Generative Tasks: Flow-based generative architectures that build the solution as a transport map guided by physics-informed kernels address complex-sampling and sign-problem scenarios in statistical physics and quantum field theory (Ihssen et al., 3 Mar 2026).

7. Open Challenges and Outlook

PIKL unifies kernel methods with the structure of physical laws, addressing limitations of both data-only and rigid physics-based models. While theoretical and algorithmic foundations are robust for linear operators and smooth domains, open directions include extension to nonlinear PDEs, more sophisticated operator learning (e.g., in neural operator frameworks), and coupling with online or sparse kernel methods for even larger scales (Doumèche et al., 2024, Doumèche et al., 2 Sep 2025).

Hybrid system identification and control, automated kernel selection, and uncertainty quantification in physical sciences are all ongoing application domains. The flexibility and rigor of PIKL, along with its growing empirical track record, position it as a central paradigm in physics-informed machine learning.