---
title: Physically Informed Neural Networks (PINNs)
url: https://www.emergentmind.com/topics/physically-informed-neural-networks-pinns
type: topic
---

# Physically Informed Neural Networks (PINNs)

Physics-informed neural networks (PINNs) parameterize the solution \(u\) of an ordinary or partial differential equation by a neural network \(u_\theta\) and train that network by penalizing violations of the governing equations, boundary and initial conditions, and, when available, observational data. In the standard formulation, automatic differentiation supplies the derivatives required by the differential operator, so the learning problem becomes optimization of a composite physics-and-data loss over collocation, boundary, and initial sets. This mesh-free paradigm has been used for forward modeling, inverse parameter identification, surrogate modeling, multiphysics coupling, and equation discovery across mechanics, fluid dynamics, radiative transfer, dispersive waves, and moving-boundary systems [2205.13748] [2410.00422] [2507.19522].

## 1. Canonical formulation and problem classes

A standard PINN seeks a neural approximation \(u_\theta(x,t)\) of a field \(u(x,t)\) satisfying a PDE or ODE together with boundary and initial conditions. In the notation used across the literature, one defines a pointwise residual \(r(x;\theta)=F[u](x;\theta)\) or, equivalently, applies a differential operator \(\mathcal N[u_\theta]\) at collocation points. The residual energy is then estimated by Monte Carlo sampling, while additional terms enforce boundary, initial, or supervised data constraints [2409.20206].

A representative composite objective is
\[
L_{\rm total}(\theta)
=
\lambda_{\rm res}L_{\rm res}(\theta)
+
\lambda_{\rm bc}L_{\rm bc}(\theta)
+
\lambda_{\rm ic}L_{\rm ic}(\theta)
+
\lambda_{\rm data}L_{\rm data}(\theta),
\]
where \(L_{\rm res}\) penalizes the PDE residual, \(L_{\rm bc}\) and \(L_{\rm ic}\) penalize boundary and initial mismatches, and \(L_{\rm data}\) is optional when scattered measurements are available [2205.13748]. In a simpler but equivalent notation, many expositions write
\[
L(\theta)=L_u+\lambda_r L_r+\lambda_b L_b,
\]
with \(L_u\) the data misfit, \(L_r\) the residual loss, and \(L_b\) the boundary loss [2410.00422].

This formulation supports both forward and inverse problems. In a forward PINN, PDE coefficients are known and one optimizes only \(\theta\). In an inverse PINN, unknown coefficients are treated as trainable variables alongside \(\theta\), so that the field and parameters are learned simultaneously by minimizing a shared data-plus-physics objective [2507.19522]. Haghighat et al. explicitly used this structure to identify Lamé parameters and elastoplastic quantities in solid mechanics, while aerodynamic and radiative-transfer studies used the same principle for hidden-field inference and parameter reconstruction [2003.02751] [2403.17470] [2009.13291].

A recurrent feature of PINN formulations is the replacement of mesh-based discretization by collocation-point sampling. That property is central to their use on irregular domains, parameterized geometries, and sparse-data settings, although the literature also shows that sampling strategy materially affects conditioning, accuracy, and the relation between residual minimization and true solution error [2205.13748] [2411.18240].

## 2. Architectural families and inductive biases

Most early PINNs use fully connected multilayer perceptrons with smooth activations such as \(\tanh\), because higher-order automatic differentiation is then well posed. The architecture, however, has diversified considerably, and the literature increasingly treats architectural choice as part of the numerical method rather than as a purely machine-learning detail [2410.00422].

| Variant | Core mechanism | Reported effect |
|---|---|---|
| Multi-network PINN | Independent subnetworks for field components or stresses | More accurate representation of field variables [2003.02751] |
| FO-PINN | First-order reformulation with compatibility residuals | \(2.2\times\) speedup; exact Dirichlet BCs via approximate distance functions [2210.14320] |
| \(\Delta\)-PINN | Laplace-Beltrami eigenfunction positional encoding | Meaningful solutions on complex shapes where traditional PINNs fail [2209.03984] |
| SetPINN | Element-wise sets plus Transformer self-attention | Unbiased, lower-variance residual-energy estimates; up to an order-of-magnitude lower rRMSE [2409.20206] |
| NeuroPINN | Variable Spiking Neuron layers with stochastic projection | High accuracy with reduced communication and energy demands [2511.06081] |

In solid mechanics, Haghighat et al. reported their best results with five independent fully connected subnetworks for \((u_x,u_y,\sigma_{xx},\sigma_{yy},\sigma_{xy})\), each using \(5\)–\(10\) hidden layers and \(20\)–\(50\) neurons per layer; they also found that \(\tanh\) gave better gradient accuracy, whereas ReLU gave poor gradient accuracy [2003.02751]. This multi-network organization is representative of a broader PINN trend: coupled physical variables are often easier to approximate when the network factorization reflects the structure of the PDE system.

FO-PINNs address a distinct issue: the cost and instability associated with second- or higher-order automatic differentiation. By augmenting the outputs with first derivatives such as \(u_x\) and \(u_y\), and adding compatibility residuals \(u_x-\partial_x u=0\) and \(u_y-\partial_y u=0\), FO-PINNs eliminate second-order differentiation from the PDE loss. On the Helmholtz example, the reported time per iteration was \(0.022\) s for a standard PINN, \(0.010\) s for FO-PINN, and \(0.0075\) s for FO-PINN with AMP, with machine-precision boundary satisfaction under the distance-function ansatz [2210.14320].

For complex geometries, \(\Delta\)-PINNs replace Euclidean coordinates by spectral coordinates built from Laplace-Beltrami eigenfunctions. On a coil with 1546 nodes and 3044 triangles, using \(K=50\) eigenfunctions and one hidden layer of 100 \(\tanh\) units, the reported NRMSE was approximately \(3.97\times 10^{-3}\), whereas the baseline PINN on Euclidean coordinates produced NRMSE approximately \(1.0\) and an almost constant solution [2209.03984]. The same framework was also used for a heat sink and a bunny, showing that geometry encoding can be decisive when topology is nontrivial.

SetPINNs modify the pointwise PINN paradigm by partitioning the domain into \(M\) elements and sampling \(K\) points per element, then processing each set with a mixer and Transformer set processor. The resulting residual-energy estimator \(\hat R\) satisfies \(E[\hat R(\theta)]=R(\theta)\), and under independent sampling the variance bound \(\mathrm{Var}[\hat R(\theta)]\le \sigma^2/(MK)\) is established in the paper [2409.20206]. The reported empirical effect was more stable convergence, lower variance across seeds, and up to an order-of-magnitude lower rRMSE than standard PINNs, Quadratic Residual networks, and PINNsFormer.

NeuroPINNs pursue a different inductive bias: sparse, event-driven computation. They replace the first one or two hidden layers with Variable Spiking Neuron layers and use a stochastic projection method to estimate spatial derivatives without differentiating through spike thresholds. On four benchmark PDEs, the measured spiking activity was roughly \(50\%\), \(52\%\), \(42\%\), and \(29\%\), implying the same percentage energy savings in synaptic operations; the reported relative percent \(L_2\) errors were \(1.04\%\), \(1.27\%\), \(2.70\%\), and \(0.34\%\) for NeuroPINN on the four tasks [2511.06081].

Architecture also determines the kernel induced in the infinite-width limit. In the kernel-task alignment analysis, smooth activations lead to rapidly decaying spectra and stronger low-frequency bias, whereas piecewise-linear activations such as ReLU yield polynomially decaying eigenvalues and better mid-frequency coverage [2307.06362]. This theoretical perspective is consistent with the empirical observation from Auto-PINN that activation choice is the single most critical hyperparameter in the tested benchmark suite [2205.13748].

## 3. Optimization, automation, and constraint enforcement

Training protocols for PINNs commonly combine first-order and quasi-Newton optimization. In solid mechanics, the reported default was Adam with learning rate \(10^{-3}\to 10^{-4}\), batch size \(64\), up to \(10\,000\) epochs, and patience \(500\) [2003.02751]. In aerodynamics, Coulaud et al. used ADAM for robustness and then L-BFGS or BFGS for fast final convergence, and reported that optimizer implementation details such as line search and tolerance strongly affect both speed and final accuracy [2403.17470]. Auto-PINN formalized a related two-phase schedule by introducing a changing point \(c\), the fraction of epochs spent with Adam before switching to L-BFGS [2205.13748].

Auto-PINN is the first systematic automated hyperparameter optimization approach for PINNs described in the supplied literature. Its search space includes width \(w\in\{8,12,16,\dots,256\}\), depth \(d\in\{3,4,\dots,10\}\), activation in \(\{\mathrm{Tanh},\mathrm{Sigmoid},\mathrm{ReLU},\mathrm{Swish}\}\), and changing point \(c\in\{0.1,0.2,0.3,0.4,0.5\}\), with fixed learning rate \(10^{-5}\) and total epochs \(10^4\), for approximately \(10{,}080\) configurations in total [2205.13748]. The reported step-by-step search uses only \(261\) trials, approximately \(2.6\%\) of the full grid, and on seven standard PDEs it achieved \(10\)–\(30\%\) lower median error with roughly \(15\%\) fewer trials than Random Search and HyperOpt, together with tighter error distributions. Auto-PINN also reported that the Adam\(\rightarrow\)L-BFGS switching point generally performs best around \(0.4\)–\(0.5\), and that one should identify “good” width bands before tuning depth [2205.13748].

A persistent issue is that standard soft-constrained losses can converge to trivial or misleading solutions. The Dynamical Boundary Constraint method addresses this by re-enforcing the network’s own previously learned trajectory at a small set of dynamic points \(\tau_j\), through an additional term
\[
L_{\rm DBC}(\theta)=\frac{1}{N_{\rm DBC}}\sum_{j=1}^{N_{\rm DBC}}\|u_\theta(\tau_j)-\hat u(\tau_j)\|^2.
\]
On the harmonic oscillator, the reported median MAE improved from approximately \(2\times 10^{-2}\) for standard PINN to approximately \(5\times 10^{-3}\) with \(N_{\rm DBC}=1\) and to approximately \(8\times 10^{-5}\) with \(N_{\rm DBC}=5\); on black-hole trajectories, the median MAE improved from approximately \(1\times 10^{-2}\) to approximately \(2\times 10^{-4}\) at \(N_{\rm DBC}=5\) [2507.21800].

A stronger remedy is to abandon penalty-only training and solve a hard-constrained optimization problem. trSQP-PINN formulates PINN training as
\[
\min_\theta \ \ell(\theta)\quad \text{s.t.}\quad c_i(\theta)=0,
\]
and applies trust-region sequential quadratic programming with quasi-Newton Hessian approximations and a feasibility pretraining phase [2409.10777]. For transport, reaction, and reaction-diffusion equations at large coefficients, the reported relative errors for trSQP-PINN were \(0.72\%\), \(2.52\%\), and \(0.66\%\), versus \(51.6\%\), \(62.6\%\), and \(68.4\%\) for a penalty method and \(48.2\%\), \(58.3\%\), and \(55.4\%\) for an augmented Lagrangian method. The paper characterizes this as a \(1\)–\(3\) orders-of-magnitude improvement in learned PDE-solution accuracy [2409.10777].

Causal and adaptive training have also emerged in transient multiphysics. For binary-alloy solidification, the moving-boundary PINN methodology alternates causal training, with temporal weights
\[
w_i=\exp\!\Bigl(-e\sum_{k=1}^{i-1}L(\tau_k)\Bigr),
\]
and adaptive weighting based on gradient statistics, while progressively freezing parts of the optimization space [2409.10910]. This is one example of a broader trend: optimizer design, loss balancing, and curriculum structure are increasingly treated as discretization-level choices rather than generic tuning parameters.

## 4. Spectral bias, consistency, and error analysis

One of the most detailed theoretical analyses of PINNs in the supplied literature derives the Neurally-Informed Equation (NIE) from the infinite-width Gaussian-process limit. In that formulation, the predicted solution satisfies
\[
f(x)+\eta_\Omega\!\int_\Omega [L f(y)-\phi(y)](L_xK)(y,x)\,dy
+\eta_\partial\!\int_{\partial\Omega}[f(y)-g(y)]K(y,x)\,dS_y=0,
\]
where \(K\) is the NNGP kernel induced by the network architecture [2307.06362]. After spectral decomposition of the kernel-augmented operator, the residual takes the form
\[
[Lf](x)-\phi(x)=\sum_k \frac{1}{1+\eta_\Omega\lambda_k}\,c_k\,\phi_k(x).
\]
This makes the PINN spectral bias explicit: components aligned with large-eigenvalue modes are suppressed first, while modes with \(\lambda_k\lesssim 1/\eta_\Omega\) dominate the remaining error [2307.06362].

A different theoretical line addresses statistical consistency and overfitting. The convergence-and-error analysis paper shows that classical unregularized training can suffer from systematic overfitting: the empirical loss may go to zero while the theoretical risk diverges. Adding ridge regularization yields risk consistency for both linear and nonlinear PDE systems, and for linear PDE systems a Sobolev-type regularization produces strong convergence to a physically consistent solution [2305.01240]. The paper further recommends always including weight decay and, for linear or mildly nonlinear PDEs, adding a Sobolev-type penalty.

The review literature organizes PINN error into approximation, estimation, and optimization components,
\[
\|u_\theta-u^*\|
\le
\varepsilon_{\text{app}}
+
\varepsilon_{\text{est}}
+
\varepsilon_{\text{opt}},
\]
and attributes many failures to poor multiscale approximation, ill-conditioning of PDE losses, insufficient convergence theory, and inadequate integration of physical information [2411.18240]. The same review reports that the Hessian of the residual loss can have eigenvalues spanning many orders of magnitude, which helps explain the slow or stalled convergence often seen in stiff or multiscale problems.

The literature does not present a single universal relation between low PINN loss and low solution error. Auto-PINN reported a near-linear log-log correlation between the smallest training loss and the true relative \(L_2\) error across its seven benchmark PDEs, making \(L_{\rm total}\) an effective search objective in that setting [2205.13748]. By contrast, broader analyses emphasize that the mismatch between PDE residual and solution error can be severe, particularly when collocation points are scattered without appropriate volume weighting or when boundary conditions are only softly enforced [2411.18240]. This tension is central to current PINN theory: residual minimization is informative, but its reliability depends on operator stability, sampling, weighting, and architecture.

Problem-specific a posteriori analyses further extend the theory. For nonlinear dispersive PDEs, rigorous \(L^2\)-error bounds are derived by exploiting classical solution stability and Grönwall arguments [2104.05584]. For radiative transfer, generalization error bounds combine training residuals, low-discrepancy quadrature error, and scattering-integral quadrature error [2009.13291]. Together, these results indicate that PINN analysis is most mature for linear or structurally stable PDE families and remains less complete for strongly nonlinear, multiscale, or high-dimensional systems.

## 5. Domain-specific formulations and applications

In solid mechanics, Haghighat et al. embedded momentum balance and constitutive relations directly into the PINN loss for linear elasticity and extended the framework to von Mises elastoplasticity [2003.02751]. On an analytical unit-square test, the PINN identified \(\lambda\) and \(\mu\) to within \(1\%\) after approximately \(2\,000\)–\(5\,000\) epochs. On high-order FEM and IGA data, the reported displacement \(L^2\)-error was below \(10^{-3}\), and transfer learning reduced the initial loss by approximately \(1\)–\(2\) orders of magnitude, with re-training converging in \(O(10^2)\) epochs. A surrogate extension using inputs \((x,y,\mu)\) achieved displacement relative error below \(2\%\) and stress error below \(5\)–\(10\%\) over \(\mu\in(0,9)\) [2003.02751].

In aerodynamics, Coulaud et al. demonstrated three non-classical uses of PINNs: parametric surrogate modeling, multiphysics coupling, and turbulence inference [2403.17470]. For a heated cavity, the parametric model used inputs \((x,y,\mu,k^f)\) and \(2500\) spatial points at \(4\) values of \(\mu\) and \(4\) values of \(k^f\); at an unseen parameter pair \((0.05,0.05)\), predictions matched a separately trained single-parameter PINN to within visually imperceptible differences. For conjugate heat transfer, jointly training fluid and solid networks with interface-continuity and flux losses reproduced FreeFEM results with MSE \(\lesssim 10^{-5}\) on velocity, temperature, and interface conditions, apart from a slightly elevated heat-equation residual near a flux discontinuity. For turbulence inference in a backward-facing step, the PINN recovered sub-percent velocity errors relative to DNS at test sections and produced the correct Reynolds-stress correlation profiles [2403.17470].

Moving-boundary multiphysics creates additional architectural and optimization demands. In binary-alloy solidification, a PINN with separate temperature, composition, and interface networks, each with five hidden layers of 100 neurons and Swish activations, was trained using alternating causal and adaptive-weighting blocks [2409.10910]. After approximately \(3\,150\) total epochs, the reported interface-position relative \(L^2\)-error was approximately \(2\%\), temperature-field MSE approximately \(2.3\times 10^{-5}\), and composition-field MSE approximately \(2.5\times 10^{-4}\). The paper emphasizes that the discontinuous jump in \(C_l\) at the moving interface and the nonlinear liquid profile were both well captured [2409.10910].

PINNs have also been deployed on high-dimensional transport problems. Mishra and Molinaro used PINNs for steady and time-dependent radiative transfer, including 3D monochromatic and polychromatic settings and a joint inverse problem for absorption and intensity [2009.13291]. In the inverse 3D steady case, the reported approximation errors were approximately \(0.65\%\) for the intensity \(u\), \(2.8\%\) for the absorption \(k\), and \(0.073\%\) for the angular moment \(G\). The reported runtimes ranged from minutes to hours on a single GPU, while avoiding the classical curse of dimensionality associated with direct discretization of the full phase space [2009.13291].

For nonlinear dispersive PDEs, PINNs have been tested on KdV-Kawahara, Camassa-Holm, and Benjamin-Ono equations [2104.05584]. Reported generalization errors include \(3.38\cdot 10^{-5}\) for a KdV single soliton, \(1.01\times 10^{-3}\) for a Kawahara single soliton, \(1.91\times 10^{-5}\) for a Camassa-Holm single soliton, and sub-percent errors for Benjamin-Ono test cases, all with \(\tanh\) networks trained by LBFGS. In a simpler heat-equation setting, a forward PINN achieved ground-truth MSE \(2.720\times 10^{-6}\), and the inverse PINN recovered \(D\approx 0.0999\pm 0.0001\) for a true diffusivity \(0.1\) [2507.19522].

## 6. Limitations, benchmarking, and research directions

PINNs are not uniformly superior to classical numerical solvers. The survey on limitations and future prospects reports that, for 1D/2D/3D Poisson problems on \([0,1]^d\), classical FEM outperforms vanilla PINNs by \(1\)–\(3\) orders of magnitude in both speed and accuracy [2411.18240]. The same source cites a nonlinear Helmholtz example in which a PINN reached max error \(\sim 10^{-5}\) in wall time \(\sim 578\) s, whereas FEM with \(400\)–\(800\) elements reached max error \(\sim 10^{-9}\) in \(\sim 8\) s, and a viscous Burgers example in which PINN error was \(\sim 10^{-3}\) in \(\sim 1800\) s versus FEM error \(\sim 10^{-5}\) in \(\sim 50\) s. These comparisons directly counter the common misconception that PINNs are generic replacements for mature PDE solvers.

The principal limitations identified across the literature are consistent. They include poor multiscale approximation and spectral bias, ill-conditioned loss landscapes, soft boundary enforcement, high cost of higher-order automatic differentiation, sensitivity to collocation design, and incomplete convergence theory for nonlinear and strongly multiscale PDEs [2411.18240] [2410.00422]. Several specialized methods target these issues individually: FO-PINNs reduce derivative order, \(\Delta\)-PINNs inject geometry, SetPINNs reduce estimator variance, DBC and trSQP-PINN improve constraint enforcement, KP-PINNs replace \(L_2\)-type residual norms by RKHS norms, and Auto-PINN automates architecture search [2210.14320] [2209.03984] [2409.20206] [2507.21800] [2409.10777] [2506.08563] [2205.13748].

What PINNs do offer, according to the supplied literature, is a unified framework for low-data inverse problems, parameterized surrogates, geometry changes, coupled physics, and settings where sparse observations must be combined with governing equations [2403.17470] [2003.02751] [2410.00422]. Transfer learning, parametric inputs, and operator-aware architectures have been particularly effective in these regimes. This suggests that PINNs are best understood not as direct competitors to high-order solvers on well-posed fixed-grid forward problems, but as differentiable scientific models occupying the interface between simulation, inference, and data assimilation.

Open directions recur across the sources. They include adaptive or residual-based sampling, stronger analysis of ill-conditioning and convergence, better loss-balancing strategies, integration with classical numerical methods, domain decomposition, extension beyond simple MLP backbones, irregular or adaptive collocation for RKHS-based formulations, and hardware-aware implementations, including neuromorphic deployment for spiking variants [2205.13748] [2411.18240] [2506.08563] [2511.06081]. The field’s trajectory is therefore methodological rather than merely applicative: PINNs are evolving through increasingly explicit numerical analysis of architecture, optimization, geometry, and operator structure.

Source: https://www.emergentmind.com/topics/physically-informed-neural-networks-pinns