Physics-Informed Neural Networks and Radial Basis Functions for PDEs with Dirac Delta Sources

Abstract: Physics-Informed Neural Networks (PINNs) are a machine learning method for solving forward and inverse Partial Differential Equations (PDEs). When applied to PDEs with Dirac delta functions in the forcing terms, boundary conditions, or initial conditions, PINNs require approximating them with smooth surrogate functions, a practice that can introduce significant modeling errors. In this work, we exploit the interpretation of PINNs as Residual Least Squares (RLS) methods and show that this perspective enables direct treatment of Dirac delta terms by integrating the weak-form equation. Among RLS formulations other than PINN, we focus on the Radial Basis Function (RBF) expansion (also known as a single-layer RBF Network). We show that while integrating out the Dirac delta in PINNs causes residuals to fail to converge to zero, RBF-RLS consistently provides good forward and inverse solutions to transport problems. We explain this finding using the Neural Tangent Kernel (NTK) theory. We test both approaches on linear PDEs that represent groundwater flow and transport in porous media and rivers. We solve inverse problems to fit synthetic data, noisy synthetic data, and real-world measurements.

• The paper develops a discrete weak-form RLS framework that integrates Dirac delta sources directly, avoiding the need for mollification.
• It demonstrates that RBF–RLS methods achieve significantly lower errors in forward and inverse problems compared to traditional PINNs.
• NTK theory is used to explain PINNs' limitations for singular sources, underscoring the efficacy of local kernel approaches for such PDEs.

Physics-Informed Neural Networks and Radial Basis Functions for PDEs with Dirac Delta Sources

Introduction and Problem Context

This paper systematically investigates the limitations of vanilla Physics-Informed Neural Networks (PINNs) and the advantages of Radial Basis Function Residual Least Squares (RBF–RLS) approaches for solving partial differential equations (PDEs) with Dirac delta source terms. Dirac delta distributions arise naturally in advection–diffusion equations with localized or impulsive sources, representative of many applications in groundwater flow, solute transport in porous media, and hydrological tracer experiments. Traditional PINN formulations require mollification of Dirac deltas, introducing substantial modeling errors. The authors revisit the minimization of PDE residuals from the lens of weighted residual and RLS methods, developing a framework for including singular source terms via direct, analytical integration in weak form.

Methodological Advances

The core methodological contributions include the discrete weak-form RLS framework for PDEs with Dirac delta terms, which can flexibly accommodate both DNN-based (PINN) and RBF-based approximators. While PINNs, formulated as overdetermined least-squares collocation, are well-established for smooth problems, their convergence properties degrade substantially for singular source terms. By contrast, RBF–RLS—operating with fixed centroids and shape parameters—yields a linear system for the expansion coefficients when both operator and basis are linear.

Direct integration of Dirac source terms at the residual level enables the avoidance of mollifiers. The RBF–RLS system can thus resolve singularities with arbitrarily localized kernels as the number of basis functions increases, an advantage unattainable by increasing neural network width in PINNs. Analytical gradients and Hessians (with respect to physical or model parameters) further facilitate efficient solution of inverse problems.

Numerical Experiments: Forward and Inverse Problems

ADE with Instantaneous Point Source

For a canonical advection–dispersion equation (ADE) with a point source initial condition, the RBF–RLS method accurately tracks the analytic propagation of the injected plume for $v=0.7$, $D=0.008$, and source at $x_\text{ip}=0.1$. Notably, as shown in the solution (Figure 1), the RMSE relative to the analytical solution falls below 0.1% at late times except near the injection, where the sharp initial profile challenges the basis representation.

Figure 1: RBF–RLS forward solution of the ADE with an instantaneous point release, demonstrating excellent agreement with the analytic solution except for localized errors in the early-time near-source region.

Contrastingly, the PINN approach fails to converge to an acceptable solution, with residuals stagnating regardless of network width or hyperparameter tuning.

ADE with Step Boundary: PINN vs. RBF–RLS

A second scenario (ADE with step boundary input) benchmarks both methods on a problem with sharp but not singular features. For equivalent numbers of trainable parameters, both methods achieve global RMSE $<5\times10^{-3}$, but RBF–RLS demonstrates lower prediction error at the measurement boundary, resulting in superior parameter recovery in the inverse problem (Figure 2).

Figure 2: Forward and inverse solution comparison for the ADE with a step boundary, showing that RBF–RLS slightly outperforms PINN at the measurement location and yields more accurate parameter estimation.

Mobile–Immobile Exchange Model

Applied to first-order mobile–immobile exchange systems calibrated to real tracer experiment data (Antietam Creek, 1969), RBF–RLS resolves the coupled PDE system efficiently and accurately. Evaluation of optimization algorithms (Nelder–Mead, L-BFGS-B, and trust-exact with analytical derivatives) indicates that Newton-type methods leveraging Hessians reach globally optimal parameter estimates with lower error and greater consistency (Figure 3).

Figure 3: Evolution of RMSE during RBF–RLS-based parameter estimation for a mobile–immobile exchange model, indicating robust and rapid convergence with second-order optimization.

Diffusion Equation with Point Source Forcing

For a standard diffusion equation with point source forcing and homogeneous boundary/initial conditions, the PINN loss landscape exhibits distinctive pathology: the losses associated with Dirac delta constraints decrease linearly, while all other loss components (boundary, initial, bulk PDE) quickly plateau, in concordance with the neural tangent kernel (NTK) analysis. In contrast, RBF–RLS attains errors an order of magnitude smaller (Figure 4).

Figure 4: Comparison of PINN and RBF–RLS for the diffusion equation with point source forcing. Left: Loss evolution in PINN highlighting stagnation of all but the singular loss term. Right: RBF–RLS achieves lower pointwise errors throughout the domain.

Theoretical Analysis: NTK Perspective

The behavior observed in PINN losses for singular source problems is rigorously justified using NTK theory. In the infinite-width limit, residuals at the Dirac location remain globally coupled to nearby residuals due to the nonlocal kernel structure. While increasing DNN width increases representational capacity for smooth functions, it does not decouple the influence of singular sources—the residuals cannot be locally minimized, and losses do not converge to zero. This formally explains the empirical stagnation observed in training.

By contrast, the (compactly supported or rapidly decaying) kernels induced by RBFs progressively localize as their width parameter decreases with increasing expansion size, permitting the independent minimization of the singular and bulk residuals—hence enabling convergence to the true distributional solution.

Implications and Future Directions

The results have broad implications for choosing function approximators in scientific machine learning for PDEs with singular data. The RBF–RLS framework provides consistent and efficient recovery of both forward and inverse solutions, is robust to prior parameter guesses, and admits closed-form gradients/Hessians for optimization. PINNs, under the vanilla formulation and without further architectural modification, exhibit inherent limitations for these tasks.

From a theoretical standpoint, the study demonstrates that representational universality (as guaranteed by neural network approximation theorems) is insufficient by itself for correct numerical solution of PDEs with singularities—training dynamics, induced kernel structure, and residual coupling must be considered. Practically, the work suggests that RBF–RLS and related local kernel methods should be favored for advection–diffusion problems with sharp/impulsive sources, especially at moderate dimensionality.

As an outlook, it would be worthwhile to explore the integration of more expressive PINN architectures (e.g., Fourier features, domain decomposition, adaptive sampling) designed to mitigate kernel coupling, or to hybridize RBF and deep neural representations. Furthermore, the extension of direct singular source integration to nonlinear and high-dimensional equations could enhance the toolkit available for parameter estimation, uncertainty quantification, and data assimilation in physical systems.

Conclusion

The paper establishes the limitations of PINNs for PDEs with Dirac delta sources in both theory and practice and demonstrates that RBF–RLS methods, when formulated to integrate singular sources directly, achieve superior accuracy and reliability for both forward modeling and parameter estimation tasks involving advection–diffusion equations. The findings clarify the consequences of loss landscape structure for learning with singularities and delineate practical recommendations for scientific ML approaches to PDEs with impulsive sources.