Physics-informed machine learning as a kernel method

Abstract: Physics-informed machine learning combines the expressiveness of data-based approaches with the interpretability of physical models. In this context, we consider a general regression problem where the empirical risk is regularized by a partial differential equation that quantifies the physical inconsistency. We prove that for linear differential priors, the problem can be formulated as a kernel regression task. Taking advantage of kernel theory, we derive convergence rates for the minimizer of the regularized risk and show that it converges at least at the Sobolev minimax rate. However, faster rates can be achieved, depending on the physical error. This principle is illustrated with a one-dimensional example, supporting the claim that regularizing the empirical risk with physical information can be beneficial to the statistical performance of estimators.

• The paper introduces a reformulation that recasts physics-informed machine learning as a kernel regression framework by integrating PDE constraints.
• The method employs a Sobolev norm and eigenvalue analysis to quantify convergence rates, leveraging the structure of physical laws.
• Numerical illustrations demonstrate effective capturing of physical phenomena and improved convergence compared to standard techniques.

Physics-informed Machine Learning as a Kernel Method

Introduction to Physics-informed Machine Learning

Physics-informed machine learning (PIML) integrates prior physical knowledge expressed via partial differential equations (PDEs) with empirical data. This approach aligns with the hybrid modeling paradigm, where estimation tasks benefit from the analytical rigor of physics while leveraging the expansive learning capacity of data-driven methods. The standard empirical risk minimization is augmented with a regularization term that embodies the physical laws in the form of a PDE penalty.

Reformulation as a Kernel Method

The paper introduces a reformulation of the PIML problem as a kernel method by demonstrating that the empirical risk, regularized with a PDE constraint, parallels the optimization problem in a function space characterized by a specific kernel. Considering linear differential operators, the solution space is endowed with a Sobolev norm, forming the foundation of a reproducing kernel Hilbert space (RKHS).

Figure 1: Illustration of a function's extension from $H^s(\Omega)$ to $H^s_{\mathrm{per}}([-2L,2L]^d)$.

Here, the kernel encapsulates both the dataset's manifold structure and the PDE's constraints, effectively transforming the machine learning problem into a structured kernel regression task. The derived kernel is specified in terms of the eigenfunctions and eigenvalues of an operator dependent on the PDE, promoting efficient numerical evaluation by leveraging properties of kernel-based learning.

Convergence Rates and Eigenvalue Analysis

The convergence rate of the estimator, $\hat{f}_n$, is hinged on the eigenvalues of the associated integral operator, $L_K$. The severity of this eigenvalue spectrum dictates the effective dimension of the kernel, which in turn influences the estimator's convergence speed. Notably, incorporating PDE-induced regularization potentially accelerates convergence beyond classical Sobolev rates by exploiting the structural insights offered by the solution manifolds of the PDE.

Numerical Illustration and Analysis

Employing an instance where the physical model has $\mathscr{D} = \frac{d}{dx}$ and dimension $d=1$, we elucidate the practical impact of this framework. The derived kernel formula reflects the synthesis of the least-squares component and PDE regularization, yielding superior learning dynamics as evidenced by empirical convergence rates.

Figure 2: Error bounds illustrating the superiority of model convergence with PDE-based regularization.

This example underscores the utility of PIML in accurately capturing phenomena where the physical model either perfectly mirrors the system (achieving parametric rates) or deviates minimally (still ensuring improved rates over standard techniques).

Conclusion and Future Directions

The exploration of PIML through the lens of kernel methods not only enhances the theoretical understanding of hybrid modeling but also offers computational pathways for leveraging physical laws in learning. Future directions will involve computational strategies for broader PDE types and exploration of kernels corresponding to more complex differential systems, aiming to generalize the methodology to encompass non-linear PDEs. The ultimate objective is to establish robust and efficient learning paradigms capable of integrating comprehensive physical knowledge into the statistical modeling framework.