Physics-Informed Networks

• Physics-informed networks are advanced neural networks that incorporate explicit physical laws and governing differential equations to achieve accurate modeling.
• They merge observational data with physics-based constraints by integrating data-fidelity, PDE residual, and boundary condition losses into the training process.
• Implemented with feed-forward architectures and adaptive sampling strategies, they effectively solve forward and inverse problems across diverse scientific domains.

Physics-informed networks are a class of machine learning models that explicitly encode physical law, typically in the form of differential equations, constraints, or conservation principles, directly into the loss function and/or architectural bias of neural networks. The paradigm was established with the introduction of Physics-Informed Neural Networks (PINNs), which have become foundational across scientific machine learning for solving forward and inverse problems related to partial differential equations (PDEs), ordinary differential equations (ODEs), stochastic differential equations, and variational formulations. By enforcing governing equations and boundary/initial conditions as soft penalties during training, these networks merge observational data with physics-based constraints, yielding models that are data-efficient, robust, and physically consistent.

1. Mathematical Formulation and Loss Construction

Let $u(x,t;\theta)$ denote a fully-connected neural network with parameters $\theta$ that approximates the solution of a PDE. The residual is defined as

$$r(x, t; \theta) = \mathcal{N}[u(x, t; \theta)] - f(x, t)$$

where $\mathcal{N}[\cdot]$ is a (possibly nonlinear) differential operator and $f$ is a known source term. The composite loss minimized during training is

$$\mathcal{L}(\theta) = \mathcal{L}_{\text{data}}(\theta) + \mathcal{L}_{\text{PDE}}(\theta) + \mathcal{L}_{\text{BC/IC}}(\theta)$$

with explicit terms:

• Data-fidelity: $$\mathcal{L}_{\text{data}} = \frac{1}{N_{\text{data}}} \sum_{i=1}^{N_{\text{data}}} |u(x_i, t_i; \theta) - u_i|^2$$
• PDE-residual: $$\mathcal{L}_{\text{PDE}} = \frac{1}{N_c} \sum_{j=1}^{N_c} |r(x_c^{(j)}, t_c^{(j)}; \theta)|^2$$
• Boundary/Initial-condition:

$$\mathcal{L}_{\text{BC/IC}} = \frac{1}{N_b} \sum_{k=1}^{N_b} |u(x_b^{(k)}, t_b^{(k)}; \theta) - g_b(x_b^{(k)}, t_b^{(k)})|^2 + \frac{1}{N_i} \sum_{\ell=1}^{N_i} |u(x_i^{(\ell)}, 0; \theta) - h(x_i^{(\ell)})|^2$$

In practice, weights $w_{\text{data}}, w_{\text{PDE}}, w_{\text{BC/IC}}$ balance these terms depending on problem regime and data availability. The residuals and gradients are efficiently computed by automatic differentiation.

2. Network Architectures, Training Protocols, and Adaptive Strategies

PINNs are commonly implemented as fully-connected feed-forward DNNs with 4–8 hidden layers and 20–200 neurons per layer, using smooth activation functions (tanh, sin) to facilitate the computation of high-order derivatives. Input and output normalization to $\theta$0 is standard for stable optimization. Collocation (residual-enforcing) points are sampled over the spatiotemporal domain via uniform or Latin-hypercube designs, with adaptive strategies focusing points where the residual is large.

Optimization is staged: Adam optimizer is used with learning rates in the $\theta$1–$\theta$2 range, followed by refinement with L-BFGS for high-precision convergence. Advanced adaptive weighting methods include:

• NTK-based weighting: loss weights are adjusted based on the spectrum of the Neural Tangent Kernel.
• Self-adaptive soft-attention: per-point, trainable weights update via a min–max or residual-based scheme.
• Residual-based attention: sample weights updated proportionally to the local residual magnitude.

3. Extensions: Domain Decomposition, Stochasticity, and Beyond

3.1 Domain Decomposition Approaches

• CPINN (Conservative PINN): The domain is split into subdomains $\theta$3, with local DNNs $\theta$4, and continuity of both the solution and flux is imposed at subdomain interfaces:

$\theta$5

• XPINN: Only residual-continuity is enforced at interfaces, enabling parallel-in-time/or space decomposition and reduced coupling between domains.
• hp-VPINN: Variational formulation where the trial space is the DNN and test functions are piecewise Legendre polynomials, yielding a global energy minimization.

3.2 Advanced Losses and Specialized Operators

• Causality-respecting losses: Residuals are temporally weighted to enforce causality.
• Stacked domain decomposition: Sequential subdomain training with tolerance-based stopping and transfer learning initializations.

3.3 Stochastic, Fractional, and Nonlocal PINNs

• PI-GANs: Combines GANs with physics-informed training to enforce stochastic differential equation residuals.
• fPINNs: Handles fractional PDEs by decomposing residuals: integer-order terms are differentiated automatically, fractional derivatives are handled via numerical quadrature with learnable orders.

4. Data-Driven Discovery and Governing Equation Identification

PINNs are applied for data-driven discovery of underlying governing laws from time series, such as multi-step ODE inference:

$\theta$6

where a DNN prior on $\theta$7 is trained to minimize the multistep error, followed by symbolic-regression (e.g., PySR) to extract explicit physical models. Demonstrated examples include learning glycolytic oscillator dynamics and recovering sparse parametric forms with relative errors down to $\theta$8–$\theta$9 (Raissi et al., 2024).

5. Theoretical Properties, Convergence, and Limitations

Rigorous analysis has established that for second-order elliptic/parabolic PDEs, as the number of collocation points $$r(x, t; \theta) = \mathcal{N}[u(x, t; \theta)] - f(x, t)$$0 and the network width increases, PINN minimizers converge in $$r(x, t; \theta) = \mathcal{N}[u(x, t; \theta)] - f(x, t)$$1 (and $$r(x, t; \theta) = \mathcal{N}[u(x, t; \theta)] - f(x, t)$$2 under BC/IC enforcement) to the true PDE solution (Shin et al. 2020, (Raissi et al., 2024)). Generalization error upper bounds are derived using training error and sample size, leveraging PDE-stability (Mishra & Molinaro 2023).

Despite their flexibility, PINNs are empirically less accurate than high-order numerical solvers (spectral, hp-FEM), incur high compute cost for forward problems, and struggle with high-dimensionality and stiff/chaotic systems over long times.

6. Applications, Impact, and Future Directions

PINNs and their variants have demonstrated efficacy across geophysics, fluid mechanics, plasma physics, materials science, computational biology, quantitative pharmacology, and digital twins, among others. Notable impact areas include:

• Data assimilation and inverse design, e.g., estimating material properties, identifying sources, or reconstructing initial conditions from data.
• Surrogate modeling for fast multi-query design and optimization, e.g., in parametric aerodynamics (Wassing et al., 2024).
• Equation discovery and scientific knowledge extraction (Raissi et al., 2024).

Ongoing challenges include improving accuracy to reach parity with established solvers, reducing training cost (e.g., tensor-decomposition DNNs yielding 100× speedups), developing robust adaptive sampling/weighting for multiscale PDEs, and integrating uncertainty quantification with heterogeneous and incomplete experimental data. Further theoretical progress is needed to characterize training dynamics, convergence, and generalization properties beyond the regimes explained by the Neural Tangent Kernel.

PINN Extension	Core Idea	Canonical Reference
CPINN	Conservative subdomain continuity	Jagtap et al. 2020
XPINN	Residual interface continuity	Jagtap & Karniadakis 2021
hp-VPINN	Variational/primal–dual formulation	Kharazmi et al. 2021
PI-GAN	PINN-driven GAN for stochastic SDEs	Yang et al. 2020
fPINN	Fractional PDE handling with DNN/quadrature	Pang et al. 2019

In summary, physics-informed neural networks constitute a versatile, theoretically motivated, and rapidly evolving paradigm for combining data-driven modeling with embedded physical knowledge. Their recent extensions—incorporating domain decomposition, stochasticity, variational formulations, adaptive weighting, and causal training—address critical barriers of scalability and robustness. The field remains propelled by open questions around scalability, accuracy, convergence theory, and integration with advanced methods for uncertainty quantification and data fusion, ensuring its continued relevance in scientific machine learning (Raissi et al., 2024).