Physics-Informed Neural Networks (PINNs)

Physics-Informed Neural Networks (PINNs) are a class of neural network frameworks designed to solve differential equations by embedding physical laws directly into the learning process. PINNs serve as mesh-free solvers that combine observed or synthetic data with the constraints imposed by governing equations, such as partial differential equations (PDEs), ordinary differential equations (ODEs), and associated boundary and initial conditions. Their unique feature lies in constructing the training loss so that the neural network outputs honor not only data but also the underlying physical model, enabling forward and inverse modeling in scientific and engineering applications.

1. Core Principles and Mathematical Formulation

PINNs encode physics by minimizing a composite loss that blends data fidelity and physics residual terms. Given a PDE of the form

$\mathcal{N}[u(x)] = \phi(x), \quad x \in \Omega$

with boundary conditions

$$\mathcal{B}[u(x)] = \psi(x), \quad x \in \partial\Omega$$

the neural network $u_\theta(x)$ approximates the unknown function, where $\theta$ denotes the trainable weights and biases.

The typical total loss is

$L = L_f + L_b$

where

$$L_f = \frac{1}{N_f} \sum_{x \in \Omega_f} |\mathcal{N}[u_\theta(x)] - \phi(x)|^2$$

$$L_b = \frac{1}{N_b} \sum_{x \in \Omega_b} |\mathcal{B}[u_\theta(x)] - \psi(x)|^2$$

and the derivatives needed for the PDE residuals are evaluated via automatic differentiation. This enables straightforward meshless enforcement of complex operators in combinatorial (space, time, parameter) domains.

PINNs are inherently flexible, handling scattered, noisy, or sparse data, non-standard domains, and can be tailored for both forward (predictive) and inverse (parameter estimation, equation discovery) problems.

2. Methodological Advances and Extensions

PINN methodology has diversified to address challenges such as stiffness, multiscale problems, domain complexity, and training pathologies:

• Fractional PINNs (fPINNs): Hybridize automatic differentiation (for integer-order operators) with explicit numerical discretization (e.g., Grunwald-Letnikov schemes) for fractional derivatives. This overcomes the lack of a chain rule for non-integer operators, making fPINNs applicable to anomalous diffusion and space-time fractional PDEs (Pang et al., 2018 ).
• First-Order PINNs (FO-PINNs): Rewrite high-order PDEs as coupled first-order systems, with the neural network outputting both the solution and its lower order derivatives, enforced by compatibility constraints. This reduces the computational cost and numerical instability associated with automatic differentiation of higher-order derivatives, and enables exact boundary condition imposition even for complex geometries (Gladstone et al., 2022 ).
• Domain Decomposition PINNs (CPINN, XPINN): Partition the physical domain (spatial, temporal, or parametric) into subdomains, deploying separate networks per region and enforcing coupling/interface consistency, improving scalability and parallelism for large/multiphysics problems (Raissi et al., 29 Aug 2024 ).
• Adaptive Loss Weighting: Use approaches such as neural tangent kernel (NTK) calibration, attention-based weighting, or local residual-based updates to dynamically balance the loss terms and accelerate training convergence, particularly in stiff or multiscale regimes (Raissi et al., 29 Aug 2024 ).
• Causality-Preserving and Time-Marching Training: Address issues in long-time or chaotic PDEs by sequential or curriculum-based training, ensuring earlier timesteps are accurately resolved before propagating through time (Raissi et al., 29 Aug 2024 , Wang et al., 2023 ).
• Meta-learning for Parameterized PINNs: Learn mappings from problem parameters (e.g., material coefficients) to optimal network initializations or weights, drastically speeding up training across parameter sweeps or uncertainty quantification tasks (Penwarden et al., 2021 ).

3. Applications in Science and Engineering

PINNs have been applied across a spectrum of classical and modern scientific computation challenges, particularly where data is sparse or models are partially known:

• Fluid Mechanics and CFD: Used to reconstruct 3D wake flows, supersonic bow shocks, and infer flow fields from partial experimental data (e.g., planar PIV, Schlieren photography). PINN-based inverse solvers can recover unknown boundary conditions, material properties, or even model mismatches in complex flow scenarios (Cai et al., 2021 , Coulaud et al., 26 Mar 2024 ).
• Radiative Transfer and High-Dimensional Problems: PINNs generalize to high-dimensional PDEs, such as 7D radiative transfer equations, achieving accuracy and computational efficiency through quasi-Monte Carlo collocation and empirical error control, often outperforming fully discrete methods in implementation effort and ease of parallelization (Mishra et al., 2020 ).
• Fractional and Anomalous Transport: fPINNs solve space-time-fractional advection-diffusion and anomalous transport equations by combining mesh-free discretization and scattered forcing observation, producing low errors even in multi-dimensional, data-sparse, or noisy regimes (Pang et al., 2018 ).
• Inverse and Multiphysics Coupling: PINNs assimilate sparse data to reconstruct hidden fields (e.g., permeability in biomedical flows, turbulent viscosity in RANS equations) or couple multiple physics (fluid-thermal, solidification with moving boundaries) by unifying different governing systems and interface conditions in the loss function (Coulaud et al., 26 Mar 2024 , Kathane et al., 17 Sep 2024 ).
• Chemical Process Systems: PINNs can infer states and rates in dynamic, partially-observed chemical processes, estimating "immeasurable" quantities given incomplete physical models, aiding in robust state estimation when both mechanistic knowledge and data are limited (Velioglu et al., 3 Jun 2024 ).
• Error Superresolution and Model Correction: PINNs have been used as super-resolvers—learning to predict high-fidelity corrections for reduced-order FE models, as well as explicit error estimation at mesh nodes, providing value for model adaptivity and surrogate-based acceleration (Zhuang et al., 14 Nov 2024 ).

4. Theoretical Foundations and Training Strategies

The theoretical analysis of PINNs has advanced markedly:

• Convergence and Generalization Guarantees: There are settings (notably linear, stable PDEs) where PINN minimizers provably converge to true solutions as the number of collocation points and network capacity increase. Error decomposition (approximation, estimation, optimization errors) parallels that in statistical learning, with more difficult PDEs exacerbating optimization error effects (Bai et al., 2021 , Raissi et al., 29 Aug 2024 , Zhang et al., 27 Nov 2024 ).
• Error Sources and Training Pathologies: Common issues include spectral bias (slow learning of high frequencies), imbalanced gradient flow (loss terms not properly weighted), and training stagnation due to highly nonconvex loss surfaces or PDE-induced ill-conditioning. Remedies include random Fourier feature embeddings, gradient-adaptive loss weighting, hard constraint enforcement, and preconditioning via operator-based analysis (Wang et al., 2023 , Zhang et al., 27 Nov 2024 ).
• Optimizer and Architecture Considerations: Second-order optimizers (e.g., L-BFGS) are often essential for complex PDEs. Model-aware neural architecture search (NAS) and meta-learning further improve performance and scalability, especially in multi-query (parametric or UQ) scenarios (Wang et al., 2022 , Penwarden et al., 2021 ).
• Robust Hard-Constrained Methods: Trust-region sequential quadratic programming (trSQP-PINN) and hard-constrained architectures markedly improve robustness versus penalty/augmented Lagrangian or purely soft-constrained methods, particularly in challenging PDEs and with large coefficients (Cheng et al., 16 Sep 2024 ).

5. Limitations, Open Challenges, and Future Directions

Despite their promise, PINNs present notable limitations:

• Multiscale and Stiff Problem Limitations: Neural network spectral bias and the inherent ill-conditioning of PDE residual losses cause difficulties with sharp gradients, localized phenomena, or high-frequency solutions. This leads to inaccurate solutions without special treatment (e.g., adaptive collocation, NAS, or hybridization with classical methods) (Zhang et al., 27 Nov 2024 ).
• Lack of General Theory and Automation: Unlike finite elements or finite volumes, there are not yet general guarantees for error and convergence across nonlinear or irregular domains. Automated architecture and hyperparameter search remains an active area of research (Raissi et al., 29 Aug 2024 , Wang et al., 2022 ).
• Physical Information Integration: Often, boundary and initial conditions are only softly imposed, which can result in physically inconsistent solutions, especially in sensitive inverse or extrapolative tasks (Zhang et al., 27 Nov 2024 ). Improved methods for “hard” constraint enforcement and incorporation of a priori physics are needed.
• Robustness to Noise and Model Misspecification: PINNs may fail under non-Gaussian measurement noise; robustification via jointly trained noise models or energy-based likelihoods is an active development (Pilar et al., 2022 ). For complex systems with incomplete or incorrect physics, discrepancy modeling with auxiliary DNNs and Bayesian or ensemble uncertainty quantification offers enhanced reliability (Zou et al., 2023 ).
• Scalability and Training Efficiency: For large-scale, high-dimensional, or high-fidelity applications, computational cost remains significant. Domain decomposition, parallelization, surrogate modeling, and meta-learning for transferability are effective strategies but require further maturation.

Tables and formulas included in the referenced materials provide concrete illustrations for loss construction, hybrid operator treatment, and example error control mechanisms.

6. Impact and Outlook

PINNs have established themselves as a unifying tool in scientific machine learning, enabling solution of forward and inverse problems in scenarios that were previously intractable using traditional techniques alone. Their mesh-free, data-integrating, and physically-consistent formulation has led to significant advances in computational fluid dynamics, anomalous transport, process systems, model error quantification, and equation discovery. Future progress will likely depend on:

• Enhanced training and optimization methods to overcome pathologies in loss landscapes.
• Architectural improvements and meta-learning to enable robust, fast many-query capability.
• Closer integration with classical numerical strategies for multiscale and stiff regimes.
• Rigorous, user-friendly frameworks for hard constraint enforcement and uncertainty quantification.
• Broader deployment in real-world, data-constrained, or heterogeneously-physical scientific and industrial applications.

As research continues, PINNs and their extensions are poised to become central tools for interpretable, robust, and flexible scientific computing at the intersection of data and theory.