Physically Informed Neural Networks (PINNs)

• Physically Informed Neural Networks (PINNs) are models that approximate differential equation solutions by embedding physics laws and boundary conditions into neural network training.
• They leverage composite loss functions and mesh-free collocation techniques to enforce PDEs, enabling applications in multiphysics, inverse problems, and parameter identification.
• Variants like FO-PINN, Δ-PINN, and SetPINN, along with adaptive optimization strategies, enhance accuracy and stability by addressing issues like ill-conditioning and spectral bias.

Physics-informed neural networks (PINNs) parameterize the solution $u$ of an ordinary or partial differential equation by a neural network $u_\theta$ and train that network by penalizing violations of the governing equations, boundary and initial conditions, and, when available, observational data. In the standard formulation, automatic differentiation supplies the derivatives required by the differential operator, so the learning problem becomes optimization of a composite physics-and-data loss over collocation, boundary, and initial sets. This mesh-free paradigm has been used for forward modeling, inverse parameter identification, surrogate modeling, multiphysics coupling, and equation discovery across mechanics, fluid dynamics, radiative transfer, dispersive waves, and moving-boundary systems (Wang et al., 2022, Ganga et al., 2024, Gupta et al., 19 Jul 2025).

1. Canonical formulation and problem classes

A standard PINN seeks a neural approximation $u_\theta(x,t)$ of a field $u(x,t)$ satisfying a PDE or ODE together with boundary and initial conditions. In the notation used across the literature, one defines a pointwise residual $r(x;\theta)=F[u](x;\theta)$ or, equivalently, applies a differential operator $\mathcal N[u_\theta]$ at collocation points. The residual energy is then estimated by Monte Carlo sampling, while additional terms enforce boundary, initial, or supervised data constraints (Nagda et al., 2024).

A representative composite objective is

$$L_{\rm total}(\theta) = \lambda_{\rm res}L_{\rm res}(\theta) + \lambda_{\rm bc}L_{\rm bc}(\theta) + \lambda_{\rm ic}L_{\rm ic}(\theta) + \lambda_{\rm data}L_{\rm data}(\theta),$$

where $L_{\rm res}$ penalizes the PDE residual, $L_{\rm bc}$ and $L_{\rm ic}$ penalize boundary and initial mismatches, and $u_\theta$0 is optional when scattered measurements are available (Wang et al., 2022). In a simpler but equivalent notation, many expositions write

$u_\theta$1

with $u_\theta$2 the data misfit, $u_\theta$3 the residual loss, and $u_\theta$4 the boundary loss (Ganga et al., 2024).

This formulation supports both forward and inverse problems. In a forward PINN, PDE coefficients are known and one optimizes only $u_\theta$5. In an inverse PINN, unknown coefficients are treated as trainable variables alongside $u_\theta$6, so that the field and parameters are learned simultaneously by minimizing a shared data-plus-physics objective (Gupta et al., 19 Jul 2025). Haghighat et al. explicitly used this structure to identify Lamé parameters and elastoplastic quantities in solid mechanics, while aerodynamic and radiative-transfer studies used the same principle for hidden-field inference and parameter reconstruction (Haghighat et al., 2020, Coulaud et al., 2024, Mishra et al., 2020).

A recurrent feature of PINN formulations is the replacement of mesh-based discretization by collocation-point sampling. That property is central to their use on irregular domains, parameterized geometries, and sparse-data settings, although the literature also shows that sampling strategy materially affects conditioning, accuracy, and the relation between residual minimization and true solution error (Wang et al., 2022, Zhang et al., 2024).

2. Architectural families and inductive biases

Most early PINNs use fully connected multilayer perceptrons with smooth activations such as $u_\theta$7, because higher-order automatic differentiation is then well posed. The architecture, however, has diversified considerably, and the literature increasingly treats architectural choice as part of the numerical method rather than as a purely machine-learning detail (Ganga et al., 2024).

Variant	Core mechanism	Reported effect
Multi-network PINN	Independent subnetworks for field components or stresses	More accurate representation of field variables (Haghighat et al., 2020)
FO-PINN	First-order reformulation with compatibility residuals	$u_\theta$8 speedup; exact Dirichlet BCs via approximate distance functions (Gladstone et al., 2022)
$u_\theta$9-PINN	Laplace-Beltrami eigenfunction positional encoding	Meaningful solutions on complex shapes where traditional PINNs fail (Costabal et al., 2022)
SetPINN	Element-wise sets plus Transformer self-attention	Unbiased, lower-variance residual-energy estimates; up to an order-of-magnitude lower rRMSE (Nagda et al., 2024)
NeuroPINN	Variable Spiking Neuron layers with stochastic projection	High accuracy with reduced communication and energy demands (Garg et al., 8 Nov 2025)

In solid mechanics, Haghighat et al. reported their best results with five independent fully connected subnetworks for $u_\theta(x,t)$0, each using $u_\theta(x,t)$1–$u_\theta(x,t)$2 hidden layers and $u_\theta(x,t)$3–$u_\theta(x,t)$4 neurons per layer; they also found that $u_\theta(x,t)$5 gave better gradient accuracy, whereas ReLU gave poor gradient accuracy (Haghighat et al., 2020). This multi-network organization is representative of a broader PINN trend: coupled physical variables are often easier to approximate when the network factorization reflects the structure of the PDE system.

FO-PINNs address a distinct issue: the cost and instability associated with second- or higher-order automatic differentiation. By augmenting the outputs with first derivatives such as $u_\theta(x,t)$6 and $u_\theta(x,t)$7, and adding compatibility residuals $u_\theta(x,t)$8 and $u_\theta(x,t)$9, FO-PINNs eliminate second-order differentiation from the PDE loss. On the Helmholtz example, the reported time per iteration was $u(x,t)$0 s for a standard PINN, $u(x,t)$1 s for FO-PINN, and $u(x,t)$2 s for FO-PINN with AMP, with machine-precision boundary satisfaction under the distance-function ansatz (Gladstone et al., 2022).

For complex geometries, $u(x,t)$3-PINNs replace Euclidean coordinates by spectral coordinates built from Laplace-Beltrami eigenfunctions. On a coil with 1546 nodes and 3044 triangles, using $u(x,t)$4 eigenfunctions and one hidden layer of 100 $u(x,t)$5 units, the reported NRMSE was approximately $u(x,t)$6, whereas the baseline PINN on Euclidean coordinates produced NRMSE approximately $u(x,t)$7 and an almost constant solution (Costabal et al., 2022). The same framework was also used for a heat sink and a bunny, showing that geometry encoding can be decisive when topology is nontrivial.

SetPINNs modify the pointwise PINN paradigm by partitioning the domain into $u(x,t)$8 elements and sampling $u(x,t)$9 points per element, then processing each set with a mixer and Transformer set processor. The resulting residual-energy estimator $r(x;\theta)=F[u](x;\theta)$0 satisfies $r(x;\theta)=F[u](x;\theta)$1, and under independent sampling the variance bound $r(x;\theta)=F[u](x;\theta)$2 is established in the paper (Nagda et al., 2024). The reported empirical effect was more stable convergence, lower variance across seeds, and up to an order-of-magnitude lower rRMSE than standard PINNs, Quadratic Residual networks, and PINNsFormer.

NeuroPINNs pursue a different inductive bias: sparse, event-driven computation. They replace the first one or two hidden layers with Variable Spiking Neuron layers and use a stochastic projection method to estimate spatial derivatives without differentiating through spike thresholds. On four benchmark PDEs, the measured spiking activity was roughly $r(x;\theta)=F[u](x;\theta)$3, $r(x;\theta)=F[u](x;\theta)$4, $r(x;\theta)=F[u](x;\theta)$5, and $r(x;\theta)=F[u](x;\theta)$6, implying the same percentage energy savings in synaptic operations; the reported relative percent $r(x;\theta)=F[u](x;\theta)$7 errors were $r(x;\theta)=F[u](x;\theta)$8, $r(x;\theta)=F[u](x;\theta)$9, $\mathcal N[u_\theta]$0, and $\mathcal N[u_\theta]$1 for NeuroPINN on the four tasks (Garg et al., 8 Nov 2025).

Architecture also determines the kernel induced in the infinite-width limit. In the kernel-task alignment analysis, smooth activations lead to rapidly decaying spectra and stronger low-frequency bias, whereas piecewise-linear activations such as ReLU yield polynomially decaying eigenvalues and better mid-frequency coverage (Seroussi et al., 2023). This theoretical perspective is consistent with the empirical observation from Auto-PINN that activation choice is the single most critical hyperparameter in the tested benchmark suite (Wang et al., 2022).

3. Optimization, automation, and constraint enforcement

Training protocols for PINNs commonly combine first-order and quasi-Newton optimization. In solid mechanics, the reported default was Adam with learning rate $\mathcal N[u_\theta]$2, batch size $\mathcal N[u_\theta]$3, up to $\mathcal N[u_\theta]$4 epochs, and patience $\mathcal N[u_\theta]$5 (Haghighat et al., 2020). In aerodynamics, Coulaud et al. used ADAM for robustness and then L-BFGS or BFGS for fast final convergence, and reported that optimizer implementation details such as line search and tolerance strongly affect both speed and final accuracy (Coulaud et al., 2024). Auto-PINN formalized a related two-phase schedule by introducing a changing point $\mathcal N[u_\theta]$6, the fraction of epochs spent with Adam before switching to L-BFGS (Wang et al., 2022).

Auto-PINN is the first systematic automated hyperparameter optimization approach for PINNs described in the supplied literature. Its search space includes width $\mathcal N[u_\theta]$7, depth $\mathcal N[u_\theta]$8, activation in $\mathcal N[u_\theta]$9, and changing point $$L_{\rm total}(\theta) = \lambda_{\rm res}L_{\rm res}(\theta) + \lambda_{\rm bc}L_{\rm bc}(\theta) + \lambda_{\rm ic}L_{\rm ic}(\theta) + \lambda_{\rm data}L_{\rm data}(\theta),$$0, with fixed learning rate $$L_{\rm total}(\theta) = \lambda_{\rm res}L_{\rm res}(\theta) + \lambda_{\rm bc}L_{\rm bc}(\theta) + \lambda_{\rm ic}L_{\rm ic}(\theta) + \lambda_{\rm data}L_{\rm data}(\theta),$$1 and total epochs $$L_{\rm total}(\theta) = \lambda_{\rm res}L_{\rm res}(\theta) + \lambda_{\rm bc}L_{\rm bc}(\theta) + \lambda_{\rm ic}L_{\rm ic}(\theta) + \lambda_{\rm data}L_{\rm data}(\theta),$$2, for approximately $$L_{\rm total}(\theta) = \lambda_{\rm res}L_{\rm res}(\theta) + \lambda_{\rm bc}L_{\rm bc}(\theta) + \lambda_{\rm ic}L_{\rm ic}(\theta) + \lambda_{\rm data}L_{\rm data}(\theta),$$3 configurations in total (Wang et al., 2022). The reported step-by-step search uses only $$L_{\rm total}(\theta) = \lambda_{\rm res}L_{\rm res}(\theta) + \lambda_{\rm bc}L_{\rm bc}(\theta) + \lambda_{\rm ic}L_{\rm ic}(\theta) + \lambda_{\rm data}L_{\rm data}(\theta),$$4 trials, approximately $$L_{\rm total}(\theta) = \lambda_{\rm res}L_{\rm res}(\theta) + \lambda_{\rm bc}L_{\rm bc}(\theta) + \lambda_{\rm ic}L_{\rm ic}(\theta) + \lambda_{\rm data}L_{\rm data}(\theta),$$5 of the full grid, and on seven standard PDEs it achieved $$L_{\rm total}(\theta) = \lambda_{\rm res}L_{\rm res}(\theta) + \lambda_{\rm bc}L_{\rm bc}(\theta) + \lambda_{\rm ic}L_{\rm ic}(\theta) + \lambda_{\rm data}L_{\rm data}(\theta),$$6–$$L_{\rm total}(\theta) = \lambda_{\rm res}L_{\rm res}(\theta) + \lambda_{\rm bc}L_{\rm bc}(\theta) + \lambda_{\rm ic}L_{\rm ic}(\theta) + \lambda_{\rm data}L_{\rm data}(\theta),$$7 lower median error with roughly $$L_{\rm total}(\theta) = \lambda_{\rm res}L_{\rm res}(\theta) + \lambda_{\rm bc}L_{\rm bc}(\theta) + \lambda_{\rm ic}L_{\rm ic}(\theta) + \lambda_{\rm data}L_{\rm data}(\theta),$$8 fewer trials than Random Search and HyperOpt, together with tighter error distributions. Auto-PINN also reported that the Adam$$L_{\rm total}(\theta) = \lambda_{\rm res}L_{\rm res}(\theta) + \lambda_{\rm bc}L_{\rm bc}(\theta) + \lambda_{\rm ic}L_{\rm ic}(\theta) + \lambda_{\rm data}L_{\rm data}(\theta),$$9L-BFGS switching point generally performs best around $L_{\rm res}$0–$L_{\rm res}$1, and that one should identify “good” width bands before tuning depth (Wang et al., 2022).

A persistent issue is that standard soft-constrained losses can converge to trivial or misleading solutions. The Dynamical Boundary Constraint method addresses this by re-enforcing the network’s own previously learned trajectory at a small set of dynamic points $L_{\rm res}$2, through an additional term

$L_{\rm res}$3

On the harmonic oscillator, the reported median MAE improved from approximately $L_{\rm res}$4 for standard PINN to approximately $L_{\rm res}$5 with $L_{\rm res}$6 and to approximately $L_{\rm res}$7 with $L_{\rm res}$8; on black-hole trajectories, the median MAE improved from approximately $L_{\rm res}$9 to approximately $L_{\rm bc}$0 at $L_{\rm bc}$1 (Martínez-Esteban et al., 29 Jul 2025).

A stronger remedy is to abandon penalty-only training and solve a hard-constrained optimization problem. trSQP-PINN formulates PINN training as

$L_{\rm bc}$2

and applies trust-region sequential quadratic programming with quasi-Newton Hessian approximations and a feasibility pretraining phase (Cheng et al., 2024). For transport, reaction, and reaction-diffusion equations at large coefficients, the reported relative errors for trSQP-PINN were $L_{\rm bc}$3, $L_{\rm bc}$4, and $L_{\rm bc}$5, versus $L_{\rm bc}$6, $L_{\rm bc}$7, and $L_{\rm bc}$8 for a penalty method and $L_{\rm bc}$9, $L_{\rm ic}$0, and $L_{\rm ic}$1 for an augmented Lagrangian method. The paper characterizes this as a $L_{\rm ic}$2–$L_{\rm ic}$3 orders-of-magnitude improvement in learned PDE-solution accuracy (Cheng et al., 2024).

Causal and adaptive training have also emerged in transient multiphysics. For binary-alloy solidification, the moving-boundary PINN methodology alternates causal training, with temporal weights

$L_{\rm ic}$4

and adaptive weighting based on gradient statistics, while progressively freezing parts of the optimization space (Kathane et al., 2024). This is one example of a broader trend: optimizer design, loss balancing, and curriculum structure are increasingly treated as discretization-level choices rather than generic tuning parameters.

4. Spectral bias, consistency, and error analysis

One of the most detailed theoretical analyses of PINNs in the supplied literature derives the Neurally-Informed Equation (NIE) from the infinite-width Gaussian-process limit. In that formulation, the predicted solution satisfies

$L_{\rm ic}$5

where $L_{\rm ic}$6 is the NNGP kernel induced by the network architecture (Seroussi et al., 2023). After spectral decomposition of the kernel-augmented operator, the residual takes the form

$L_{\rm ic}$7

This makes the PINN spectral bias explicit: components aligned with large-eigenvalue modes are suppressed first, while modes with $L_{\rm ic}$8 dominate the remaining error (Seroussi et al., 2023).

A different theoretical line addresses statistical consistency and overfitting. The convergence-and-error analysis paper shows that classical unregularized training can suffer from systematic overfitting: the empirical loss may go to zero while the theoretical risk diverges. Adding ridge regularization yields risk consistency for both linear and nonlinear PDE systems, and for linear PDE systems a Sobolev-type regularization produces strong convergence to a physically consistent solution (Doumèche et al., 2023). The paper further recommends always including weight decay and, for linear or mildly nonlinear PDEs, adding a Sobolev-type penalty.

The review literature organizes PINN error into approximation, estimation, and optimization components,

$L_{\rm ic}$9

and attributes many failures to poor multiscale approximation, ill-conditioning of PDE losses, insufficient convergence theory, and inadequate integration of physical information (Zhang et al., 2024). The same review reports that the Hessian of the residual loss can have eigenvalues spanning many orders of magnitude, which helps explain the slow or stalled convergence often seen in stiff or multiscale problems.

The literature does not present a single universal relation between low PINN loss and low solution error. Auto-PINN reported a near-linear log-log correlation between the smallest training loss and the true relative $u_\theta$00 error across its seven benchmark PDEs, making $u_\theta$01 an effective search objective in that setting (Wang et al., 2022). By contrast, broader analyses emphasize that the mismatch between PDE residual and solution error can be severe, particularly when collocation points are scattered without appropriate volume weighting or when boundary conditions are only softly enforced (Zhang et al., 2024). This tension is central to current PINN theory: residual minimization is informative, but its reliability depends on operator stability, sampling, weighting, and architecture.

Problem-specific a posteriori analyses further extend the theory. For nonlinear dispersive PDEs, rigorous $u_\theta$02-error bounds are derived by exploiting classical solution stability and Grönwall arguments (Bai et al., 2021). For radiative transfer, generalization error bounds combine training residuals, low-discrepancy quadrature error, and scattering-integral quadrature error (Mishra et al., 2020). Together, these results indicate that PINN analysis is most mature for linear or structurally stable PDE families and remains less complete for strongly nonlinear, multiscale, or high-dimensional systems.

5. Domain-specific formulations and applications

In solid mechanics, Haghighat et al. embedded momentum balance and constitutive relations directly into the PINN loss for linear elasticity and extended the framework to von Mises elastoplasticity (Haghighat et al., 2020). On an analytical unit-square test, the PINN identified $u_\theta$03 and $u_\theta$04 to within $u_\theta$05 after approximately $u_\theta$06–$u_\theta$07 epochs. On high-order FEM and IGA data, the reported displacement $u_\theta$08-error was below $u_\theta$09, and transfer learning reduced the initial loss by approximately $u_\theta$10–$u_\theta$11 orders of magnitude, with re-training converging in $u_\theta$12 epochs. A surrogate extension using inputs $u_\theta$13 achieved displacement relative error below $u_\theta$14 and stress error below $u_\theta$15–$u_\theta$16 over $u_\theta$17 (Haghighat et al., 2020).

In aerodynamics, Coulaud et al. demonstrated three non-classical uses of PINNs: parametric surrogate modeling, multiphysics coupling, and turbulence inference (Coulaud et al., 2024). For a heated cavity, the parametric model used inputs $u_\theta$18 and $u_\theta$19 spatial points at $u_\theta$20 values of $u_\theta$21 and $u_\theta$22 values of $u_\theta$23; at an unseen parameter pair $u_\theta$24, predictions matched a separately trained single-parameter PINN to within visually imperceptible differences. For conjugate heat transfer, jointly training fluid and solid networks with interface-continuity and flux losses reproduced FreeFEM results with MSE $u_\theta$25 on velocity, temperature, and interface conditions, apart from a slightly elevated heat-equation residual near a flux discontinuity. For turbulence inference in a backward-facing step, the PINN recovered sub-percent velocity errors relative to DNS at test sections and produced the correct Reynolds-stress correlation profiles (Coulaud et al., 2024).

Moving-boundary multiphysics creates additional architectural and optimization demands. In binary-alloy solidification, a PINN with separate temperature, composition, and interface networks, each with five hidden layers of 100 neurons and Swish activations, was trained using alternating causal and adaptive-weighting blocks (Kathane et al., 2024). After approximately $u_\theta$26 total epochs, the reported interface-position relative $u_\theta$27-error was approximately $u_\theta$28, temperature-field MSE approximately $u_\theta$29, and composition-field MSE approximately $u_\theta$30. The paper emphasizes that the discontinuous jump in $u_\theta$31 at the moving interface and the nonlinear liquid profile were both well captured (Kathane et al., 2024).

PINNs have also been deployed on high-dimensional transport problems. Mishra and Molinaro used PINNs for steady and time-dependent radiative transfer, including 3D monochromatic and polychromatic settings and a joint inverse problem for absorption and intensity (Mishra et al., 2020). In the inverse 3D steady case, the reported approximation errors were approximately $u_\theta$32 for the intensity $u_\theta$33, $u_\theta$34 for the absorption $u_\theta$35, and $u_\theta$36 for the angular moment $u_\theta$37. The reported runtimes ranged from minutes to hours on a single GPU, while avoiding the classical curse of dimensionality associated with direct discretization of the full phase space (Mishra et al., 2020).

For nonlinear dispersive PDEs, PINNs have been tested on KdV-Kawahara, Camassa-Holm, and Benjamin-Ono equations (Bai et al., 2021). Reported generalization errors include $u_\theta$38 for a KdV single soliton, $u_\theta$39 for a Kawahara single soliton, $u_\theta$40 for a Camassa-Holm single soliton, and sub-percent errors for Benjamin-Ono test cases, all with $u_\theta$41 networks trained by LBFGS. In a simpler heat-equation setting, a forward PINN achieved ground-truth MSE $u_\theta$42, and the inverse PINN recovered $u_\theta$43 for a true diffusivity $u_\theta$44 (Gupta et al., 19 Jul 2025).

6. Limitations, benchmarking, and research directions

PINNs are not uniformly superior to classical numerical solvers. The survey on limitations and future prospects reports that, for 1D/2D/3D Poisson problems on $u_\theta$45, classical FEM outperforms vanilla PINNs by $u_\theta$46–$u_\theta$47 orders of magnitude in both speed and accuracy (Zhang et al., 2024). The same source cites a nonlinear Helmholtz example in which a PINN reached max error $u_\theta$48 in wall time $u_\theta$49 s, whereas FEM with $u_\theta$50–$u_\theta$51 elements reached max error $u_\theta$52 in $u_\theta$53 s, and a viscous Burgers example in which PINN error was $u_\theta$54 in $u_\theta$55 s versus FEM error $u_\theta$56 in $u_\theta$57 s. These comparisons directly counter the common misconception that PINNs are generic replacements for mature PDE solvers.

The principal limitations identified across the literature are consistent. They include poor multiscale approximation and spectral bias, ill-conditioned loss landscapes, soft boundary enforcement, high cost of higher-order automatic differentiation, sensitivity to collocation design, and incomplete convergence theory for nonlinear and strongly multiscale PDEs (Zhang et al., 2024, Ganga et al., 2024). Several specialized methods target these issues individually: FO-PINNs reduce derivative order, $u_\theta$58-PINNs inject geometry, SetPINNs reduce estimator variance, DBC and trSQP-PINN improve constraint enforcement, KP-PINNs replace $u_\theta$59-type residual norms by RKHS norms, and Auto-PINN automates architecture search (Gladstone et al., 2022, Costabal et al., 2022, Nagda et al., 2024, Martínez-Esteban et al., 29 Jul 2025, Cheng et al., 2024, Yang et al., 10 Jun 2025, Wang et al., 2022).

What PINNs do offer, according to the supplied literature, is a unified framework for low-data inverse problems, parameterized surrogates, geometry changes, coupled physics, and settings where sparse observations must be combined with governing equations (Coulaud et al., 2024, Haghighat et al., 2020, Ganga et al., 2024). Transfer learning, parametric inputs, and operator-aware architectures have been particularly effective in these regimes. This suggests that PINNs are best understood not as direct competitors to high-order solvers on well-posed fixed-grid forward problems, but as differentiable scientific models occupying the interface between simulation, inference, and data assimilation.

Open directions recur across the sources. They include adaptive or residual-based sampling, stronger analysis of ill-conditioning and convergence, better loss-balancing strategies, integration with classical numerical methods, domain decomposition, extension beyond simple MLP backbones, irregular or adaptive collocation for RKHS-based formulations, and hardware-aware implementations, including neuromorphic deployment for spiking variants (Wang et al., 2022, Zhang et al., 2024, Yang et al., 10 Jun 2025, Garg et al., 8 Nov 2025). The field’s trajectory is therefore methodological rather than merely applicative: PINNs are evolving through increasingly explicit numerical analysis of architecture, optimization, geometry, and operator structure.