Physics-Informed Neural Networks (PINNs)

• Physics-informed Neural Networks (PINNs) are deep learning frameworks that incorporate physical laws via differential operators, boundary, and initial conditions.
• They recast both forward and inverse problems as unified optimization tasks, coupling network parameter estimation with physical coefficient inference.
• Variants like causal, domain-decomposed, and hard-constrained PINNs enhance robustness and multiscale approximation for applications from fluid dynamics to surrogate modeling.

Physics-informed neural networks (PINNs) are neural-network surrogates trained so that the learned field satisfies a governing differential operator, boundary and initial constraints, and, when available, observational data. In PDE settings, PINNs recast forward solution and parameter inference as a single optimization problem over network parameters and, in inverse settings, unknown physical coefficients. The framework has expanded from strong-form residual minimization to weak, variational, causal, domain-decomposed, and hard-constrained formulations, but its practical development has been shaped by recurrent issues of conditioning, multiscale approximation, and the gap between residual minimization and physically correct solution error (Zhang et al., 2024, Raissi et al., 2024).

1. Mathematical formulation and problem classes

A standard PINN starts from a PDE posed on a space–time domain $\Omega\times[0,T]$,

$\mathcal{L}[u](\mathbf{x},t)=f(\mathbf{x},t),$

with boundary operator $\mathcal{B}[u](\mathbf{x},t)=b(\mathbf{x},t)$ on $\partial\Omega\times[0,T]$ and initial condition $u(\mathbf{x},0)=u_0(\mathbf{x})$. A neural ansatz $u_\theta(\mathbf{x},t)$ approximates $u$, while automatic differentiation computes the PDE residual

$$r(\mathbf{x},t;\theta)=\mathcal{L}[u_\theta](\mathbf{x},t)-f(\mathbf{x},t),$$

together with boundary and initial discrepancies (Zhang et al., 2024).

The canonical strong-form objective is a composite loss over interior collocation points, boundary points, and initial points,

$$\mathcal{L}(\theta)= \lambda_r\frac{1}{N_r}\sum_{i=1}^{N_r}\|r(\mathbf{x}_i,t_i;\theta)\|^2+ \lambda_b\frac{1}{N_b}\sum_{j=1}^{N_b}\|\mathcal{B}[u_\theta](\mathbf{x}_j,t_j)-b(\mathbf{x}_j,t_j)\|^2+ \lambda_i\frac{1}{N_i}\sum_{k=1}^{N_i}\|u_\theta(\mathbf{x}_k,0)-u_0(\mathbf{x}_k)\|^2.$$

Training commonly combines first-order optimizers such as Adam with quasi-Newton refinement such as L-BFGS; smooth activations such as $\tanh$, Swish, or sinusoidal functions are preferred when higher-order derivatives enter the residual (Zhang et al., 2024, Labeb et al., 20 Jun 2026).

The same formulation extends directly to inverse problems. Unknown physical parameters $\mathcal{L}[u](\mathbf{x},t)=f(\mathbf{x},t),$0, including coefficients, source terms, material properties, or constitutive parameters, are made trainable and optimized jointly with $\mathcal{L}[u](\mathbf{x},t)=f(\mathbf{x},t),$1. This “all-at-once” optimization is one of the defining features of PINNs: forward solution and parameter identification are not separated into different numerical pipelines but coupled through the same physics residual (Zhang et al., 2024).

2. Optimization viewpoint, approximation, and solution error

PINNs can be read as a finite-dimensional relaxation of a PDE-constrained problem: admissible solutions are attracted toward the manifold defined by the governing equations and constraints. In this view, the physics residual acts as a structured regularizer that prunes the hypothesis space toward physically consistent functions, which is especially valuable when data are sparse, indirect, or only partially observed (Labeb et al., 20 Jun 2026).

This perspective does not remove the central approximation-theoretic difficulty. The total error decomposes into approximation, estimation, and optimization components, and practical training reaches only local minima of non-convex objectives. Current theoretical analyses often rely on idealized assumptions such as infinite width, infinitesimal learning rates, or linearized NTK dynamics; these assumptions do not directly transfer to finite-width, multi-loss, multi-derivative training with Adam or L-BFGS (Zhang et al., 2024).

A recurrent misconception is that a small residual implies an accurate solution. In the notation

$\mathcal{L}[u](\mathbf{x},t)=f(\mathbf{x},t),$2

small $\mathcal{L}[u](\mathbf{x},t)=f(\mathbf{x},t),$3 does not guarantee small $\mathcal{L}[u](\mathbf{x},t)=f(\mathbf{x},t),$4 unless the underlying continuous–discrete operator pair satisfies suitable stability, consistency, and conditioning properties. This is one of the main reasons PINN training can return physically incorrect states even when residual norms appear satisfactory (Zhang et al., 2024).

The approximation bottleneck is particularly severe for multiscale and anisotropic solutions. Standard MLPs exhibit a frequency principle: during gradient descent they fit low-frequency components first and struggle with boundary layers, oscillatory modes, shocks, or thin internal layers. The illustrative function $\mathcal{L}[u](\mathbf{x},t)=f(\mathbf{x},t),$5 is a standard example in which vanilla PINNs reduce the residual while missing the small-scale component, producing large pointwise errors (Zhang et al., 2024).

3. Failure modes, conditioning, and hard constraints

A systematic diagnosis identifies three root causes of PINN limitations: poor multiscale approximation ability and ill-conditioning caused by PDE losses; insufficient exploration of convergence and error analysis, resulting in weak mathematical rigor; and inadequate integration of physical information, causing mismatch between residuals and iteration errors (Zhang et al., 2024).

Ill-conditioning is intrinsic to the derivative-based loss. PDE losses mix zeroth-, first-, and higher-order derivatives with heterogeneous magnitudes and units, so gradient components can differ by orders of magnitude. The resulting Jacobians, Hessians, and composite operators exhibit large condition numbers,

$\mathcal{L}[u](\mathbf{x},t)=f(\mathbf{x},t),$6

which slow gradient descent, destabilize multi-objective balancing, and can trap optimization in poor local minima. Nonuniform collocation can worsen the problem because residuals are then weighted by an implicit, and often undesirable, discretization induced by sampling density and local volumes (Zhang et al., 2024).

Constraint treatment is another major source of pathology. Soft boundary and initial penalties permit slight violations at training points, but even small boundary errors may have large global impact. For time-dependent problems, training on the entire interval can violate temporal causality; unsteady cylinder flow and Kármán vortex shedding are cited examples where causal training or time marching restores physically correct dynamics (Zhang et al., 2024). This suggests that PINNs are not simply mesh-free PDE solvers, but optimization procedures whose numerical behavior depends strongly on loss design and physics encoding.

Several families of hard-constrained or derivative-reduced methods target these issues directly. FO-PINNs rewrite higher-order PDEs as first-order systems with auxiliary variables, thereby avoiding second- and higher-order automatic differentiation and enabling exact boundary imposition through approximate distance functions; in the reported Helmholtz and Navier–Stokes examples, they deliver one order-of-magnitude lower validation errors than standard PINNs and about $\mathcal{L}[u](\mathbf{x},t)=f(\mathbf{x},t),$7 overall speedup relative to standard PINNs (Gladstone et al., 2022). Trust-region SQP PINNs replace soft penalties in the step computation by a hard-constrained sequential quadratic program and use the soft-constrained loss only as a merit function; the method achieves up to $\mathcal{L}[u](\mathbf{x},t)=f(\mathbf{x},t),$8–$\mathcal{L}[u](\mathbf{x},t)=f(\mathbf{x},t),$9 orders of magnitude lower errors than penalty and augmented Lagrangian baselines on the reported benchmarks (Cheng et al., 2024).

4. Major variants and extensions

Methodological diversification has largely followed the observed failure modes. Weak and variational formulations minimize energies or weak residuals rather than strong-form pointwise residuals, often improving conditioning and robustness. Conservative PINNs enforce integral balances over control volumes, domain-decomposition methods such as cPINN and XPINN partition space or space–time into local subproblems, and causal or time-marching variants enforce temporal ordering explicitly. Residual-based adaptive refinement, volume-weighted residuals, nondimensionalization, dynamic scheduling of $\mathcal{B}[u](\mathbf{x},t)=b(\mathbf{x},t)$0, Fourier-feature encodings, and coordinate transformations are all used to improve multiscale resolution and optimization balance (Zhang et al., 2024, Raissi et al., 2024).

Several extensions modify either the operator class or the training objective itself. fPINNs combine automatic differentiation for integer-order operators with numerical discretization for fractional operators, enabling forward and inverse solution of space-, time-, and space-time-fractional advection–diffusion equations in $\mathcal{B}[u](\mathbf{x},t)=b(\mathbf{x},t)$1D–$\mathcal{B}[u](\mathbf{x},t)=b(\mathbf{x},t)$2D, with reported multi-dimensional relative errors of $\mathcal{B}[u](\mathbf{x},t)=b(\mathbf{x},t)$3 to $\mathcal{B}[u](\mathbf{x},t)=b(\mathbf{x},t)$4 (Pang et al., 2018). PINNs with unknown measurement noise replace the Gaussian MSE data term by a learned energy-based likelihood over residuals, which improves robustness under non-Gaussian and biased noise (Pilar et al., 2022). THINNs replace heuristic residual penalties by thermodynamically consistent large-deviations rate functionals, so the residual penalizes improbable deviations under the fluctuation structure of the underlying stochastic system rather than a generic $\mathcal{B}[u](\mathbf{x},t)=b(\mathbf{x},t)$5 norm (Castro et al., 23 Sep 2025).

Architecture and hyperparameter search have also become a subject in their own right. Auto-PINN treats depth, width, activation, and the Adam-to-L-BFGS switching point as an explicit neural architecture search problem and argues, on its benchmark suite, that the smallest training loss is a usable proxy objective because it correlates strongly with relative $\mathcal{B}[u](\mathbf{x},t)=b(\mathbf{x},t)$6 error (Wang et al., 2022). This suggests that part of PINN instability is not exclusively analytical or physical, but also architectural.

5. Applications and empirical behavior

PINNs are now used across forward modeling, inverse problems, equation discovery, surrogate construction, and hybrid simulation. In fluid mechanics, they have been applied to incompressible and compressible flows, wake reconstruction from sparse $\mathcal{B}[u](\mathbf{x},t)=b(\mathbf{x},t)$7D2C data, supersonic flow inference from Schlieren-like and sparse pressure measurements, and multiphysics biomedical flows (Cai et al., 2021). In aerodynamics, they have been used for parametric cavity-flow surrogates, conjugate heat transfer, and RANS-based eddy-viscosity inference in a backward-facing step, where a positivity-preserving parameterization $\mathcal{B}[u](\mathbf{x},t)=b(\mathbf{x},t)$8 supports data assimilation from sparse mean-velocity measurements (Coulaud et al., 2024).

The framework is not restricted to bounded, fully specified PDE settings. For inverse problems on infinite and semi-infinite domains, a dual-network architecture combined with Gaussian and negative-exponential collocation sampling can recover both the state $\mathcal{B}[u](\mathbf{x},t)=b(\mathbf{x},t)$9 and coefficient field $\partial\Omega\times[0,T]$0 without explicit far-field boundary terms when solutions stabilize at infinity. In the reported variable-coefficient Poisson experiments, PINNs solved the inverse problem approximately $\partial\Omega\times[0,T]$1 faster than PIKANs and achieved relative error in $\partial\Omega\times[0,T]$2 of about $\partial\Omega\times[0,T]$3 versus about $\partial\Omega\times[0,T]$4 without noise (Pérez-Bernal et al., 12 Dec 2025).

Industrial and cyber-physical case studies show a different use pattern: PINNs as physics-constrained virtual sensors or latent-state estimators. In water distribution security, a PINN trained on Navier–Stokes flow past a cylinder is proposed as a surrogate sensor under false-data injection, with reported test accuracies of $\partial\Omega\times[0,T]$5 for $\partial\Omega\times[0,T]$6 and $\partial\Omega\times[0,T]$7 for $\partial\Omega\times[0,T]$8 (Falas et al., 2020). In dynamic process operations with incomplete semi-explicit DAEs, PINNs infer unmeasured algebraic states and some unmeasured differential states even when constitutive equations are withheld; in the reported CSTR study, $\partial\Omega\times[0,T]$9 remained estimable with $u(\mathbf{x},0)=u_0(\mathbf{x})$0, whereas $u(\mathbf{x},0)=u_0(\mathbf{x})$1 did not, with $u(\mathbf{x},0)=u_0(\mathbf{x})$2–$u(\mathbf{x},0)=u_0(\mathbf{x})$3 (Velioglu et al., 2024).

Computational mechanics and hybrid simulation provide further examples. For finite-element model-error approximation and superresolution, physics-informed displacement-consistency losses outperform a purely data-driven baseline; adding the displacement-consistency term improves the model-error loss by $u(\mathbf{x},0)=u_0(\mathbf{x})$4 on testing relative to the data-only case (Zhuang et al., 2024). In Fischer–Tropsch catalyst-particle transport, a PINN is embedded inside a finite-difference solver to evaluate microkinetic source terms. The central result is not merely acceleration, but that domain-knowledge-based output reparameterization is needed to recover the correct asymptotic behavior and stabilize the overall hybrid solver (Nikolaienko et al., 2024).

6. Relation to classical solvers and open problems

PINNs are best understood as complementary rather than replacement technology. Their principal strengths are the unified treatment of forward and inverse problems, direct integration of sparse data, mesh-free collocation on complex domains, and differentiable coupling across physics modules. Their principal weaknesses are lower forward-solve accuracy, higher optimization cost, sensitivity to loss balancing, soft constraint violations, and the lack of rigorous convergence guarantees in practical finite-width regimes (Zhang et al., 2024, Labeb et al., 20 Jun 2026).

Direct comparisons with established discretization methods remain sobering. For $u(\mathbf{x},0)=u_0(\mathbf{x})$5D–$u(\mathbf{x},0)=u_0(\mathbf{x})$6D Poisson problems, FEM typically achieves $u(\mathbf{x},0)=u_0(\mathbf{x})$7–$u(\mathbf{x},0)=u_0(\mathbf{x})$8 orders of magnitude lower runtime and substantially lower relative errors than vanilla PINNs at similar discretization, although the gap narrows in higher dimensions (Zhang et al., 2024). Reviews comparing PINNs with FEM/FDM/FVM/FDTD consistently position classical solvers as preferable for routine forward simulations on simple geometries or when very high numerical accuracy is required, while PINNs remain attractive when inverse design, sparse-data assimilation, differentiable surrogates, or mesh-challenged geometries dominate the problem formulation (Labeb et al., 20 Jun 2026, Ganga et al., 2024).

The current research frontier is therefore defined less by new application domains than by unresolved numerical analysis. Open problems repeatedly identified across the literature include rigorous convergence and error theory in practical mixed-optimizer settings; principled preconditioning based on operator spectra and PDE Jacobians; robust multiscale representation beyond the spectral bias of standard MLPs; reliable handling of stiff, high-Reynolds-number, and shock-dominated regimes; scalable training for high-dimensional and multi-physics systems; stronger hard-constraint integration; and benchmarking protocols that report runtime, error norms, and diagnostic conditioning statistics in a way comparable to mature numerical methods (Zhang et al., 2024, Raissi et al., 2024, Labeb et al., 20 Jun 2026). The resulting picture is not one of a single mature solver class, but of a rapidly differentiating family of physics-constrained learning algorithms whose performance depends decisively on how physics is encoded, not merely on the presence of a PDE residual in the loss.