Physics-Informed Neural Networks

• Physics-Informed Neural Networks are computational frameworks that embed physical laws into network training to solve differential equations.
• They employ loss functions combining PDE residuals, boundary, and initial condition errors to achieve robust data efficiency and physical consistency.
• Domain decomposition and advanced initialization strategies enhance convergence speed and accuracy in complex simulations.

A Physics-Informed Neural Network (PINN) framework is a computational paradigm at the intersection of scientific computing, numerical analysis, and machine learning, specifically designed to solve the forward and inverse problems associated with partial differential equations (PDEs) or other differential/integral equations by incorporating known physical laws directly into neural network-based function approximations. By enforcing the governing equations, boundary conditions, and initial conditions in the loss function of the neural network, the framework achieves data-efficiency, robust generalization, and intrinsic consistency with the underlying physics of the modeled system.

1. Core Principles and Mathematical Formulation

At its foundation, a PINN framework parameterizes the sought solution $u(\mathbf{x}, t)$ of a target PDE (possibly involving additional unknown coefficients) as $u_\theta(\mathbf{x}, t)$, with $\theta$ denoting trainable network weights and biases. The physics-informed loss function penalizes violations of the strong-form PDE:

$$\mathcal{F}(u(\mathbf{x}, t), \mathbf{x}, t) = 0, \quad (\mathbf{x}, t) \in \Omega \times [0,T]$$

alongside boundary and initial conditions:

$$u(\mathbf{x}, t) = f(\mathbf{x}, t)\ \text{on}\ \partial\Omega \times [0,T], \quad u(\mathbf{x}, 0) = g(\mathbf{x})\ \text{in}\ \Omega.$$

The standard PINN loss is a weighted sum:

$$L_{PINN}(\theta) = \lambda_1 L_{res}(\theta) + \lambda_2 L_{bc}(\theta) + \lambda_3 L_{ic}(\theta)$$

where \begin{align*} L_{res}(\theta) &= \frac{1}{N_R} \sum_{k=1}^{N_R} |\mathcal{F}(u_\theta, \mathbf{x}k^R, t_k^R)|^2 \ L{bc}(\theta) &= \frac{1}{N_B} \sum_{k=1}^{N_B} |u_\theta(\mathbf{x}k^B, t_k^B) - f(\mathbf{x}_k^B, t_k^B)|^2 \ L{ic}(\theta) &= \frac{1}{N_I} \sum_{k=1}^{N_I} |u_\theta(\mathbf{x}_k^I, 0) - g(\mathbf{x}_k^I)|^2. \end{align*}

Automatic differentiation is used to compute the required derivatives with respect to the network inputs. These losses are minimized via gradient-based optimizers, such as Adam or L-BFGS.

2. Domain Decomposition and Initialization Strategies

To address the inherent limitations of global networks (such as slow convergence in large or stiff domains and pathological interface behavior), the PINN framework can be extended with domain decomposition. The computational domain $\Omega$ is partitioned into $M$ non-overlapping subdomains $\{\Omega_i\}_{i=1}^M$, each assigned an independent neural network $u^{(i)}_{\theta_i}$. Interface penalties on overlapping boundaries enforce continuity and smoothness: \begin{align*} L_{inter}(\theta) &= \sum_{(i,j)} \frac{1}{N_{ij}} \sum_{k=1}^{N_{ij}} |u^{(i)}_{\theta_i}(\mathbf{x}_k^{ij}, t_k^{ij}) - u^{(j)}_{\theta_j}(\mathbf{x}_k^{ij}, t_k^{ij})|^2, \ L_{\nabla}(\theta) &= \sum_{(i,j)} \frac{1}{N_{ij}} \sum_{k=1}^{N_{ij}} |\nabla u^{(i)}_{\theta_i}(\mathbf{x}_k^{ij}, t_k^{ij}) - \nabla u^{(j)}_{\theta_j}(\mathbf{x}_k^{ij}, t_k^{ij})|_2^2, \ L_{PDE_g}(\theta) &= \sum_{(i,j)} \frac{1}{N_{ij}} \sum_{k=1}^{N_{ij}} |\nabla \mathcal{F}(u^{(i)}_{\theta_i}, \cdot) - \nabla \mathcal{F}(u^{(j)}_{\theta_j}, \cdot)|_2^2. \end{align*} as in the IDPINN framework (Si et al., 5 Jun 2024).

Moreover, initialization enhancement is realized by pre-training a global PINN on a small sample of the dataset to generate a good parameter initialization $\theta^0$, which is used to initialize all subdomain networks. This significantly accelerates and stabilizes convergence.

Component	Classical PINN	IDPINN/XPINN
Global Loss	PDE + BC + IC MSE	Sum over subdomains + interface
Architecture	Single neural net	$M$ neural nets
Initialization	Random	Pretrained (IDPINN-init)
Interface Terms	None	$L_{inter}$, $L_\nabla$, $L_{PDE_g}$

3. Training Algorithms and Network Architectures

Each subdomain network typically employs a fully connected feedforward topology. Example parameterizations for various problems include 5–9 hidden layers of width 20–55, and tanh activations. All subdomain networks have the same architecture to ensure seamless copying of initial weights.

The IDPINN training algorithm proceeds in two phases:

1. IDPINN-init: Train a small global PINN for $N_{init}$ steps on a modest dataset to obtain $\theta^0$.
2. IDPINN-main: Initialize each subdomain with $\theta^0$, enforce both per-domain physics and interface penalties, and train all networks jointly via Adam.

Sampling collocation points for PDE, BC, IC, and interfaces is implemented by stratified random selection or uniform meshes.

Sample Pseudocode Outline

theta0 = train_PINN_on_small_dataset(...) for i in 1..M: theta_i = theta0 for epoch in 1..N_main: for i in 1..M: compute L_{res}^i, L_{bc}^i, L_{ic}^i compute L_{inter}, L_{\nabla}, L_{PDE_g} L_total = sum over all network and interface losses update theta_1...theta_M with Adam

4. Comparative Performance and Benchmarking

Empirical studies on canonical test problems (2D Helmholtz, Poisson, heat equation, viscous Burgers) demonstrate substantial accuracy gains for domain-decomposed and initialization-enhanced PINN frameworks:

• For a 2D Helmholtz equation on $[-1,1]^2$ partitioned at $x=0$, IDPINN with full interface regularization achieves a relative $L_2$ error of $1.10 \times 10^{-2}$, compared to $1.41 \times 10^{-1}$ for classical XPINN under identical hyperparameters.
• On irregular subdomain Poisson problems, inclusion of gradient and PDE-residual gradient continuity terms consistently reduces relative $L_2$ errors by factors of $2$–$5$, with best results (IDPINN-3) producing $6.48\times 10^{-4}$.
• For time-dependent problems with spatio-temporal domain partitions, such as the heat equation on $[-1,1]\times[0,1]$ split at $t=0.5$, IDPINN-3 yields $L_2 = 3.51 \times 10^{-3}$, outperforming both classical PINN and XPINN.

Additional benchmarking shows that the IDPINN initialization step reduces total training time by up to $5 \times$ relative to domain-decomposed PINNs initialized from scratch (Si et al., 5 Jun 2024).

5. Limitations, Challenges, and Potential Extensions

Despite the demonstrated improvements, the PINN framework with domain decomposition and initialization enhancement presents several open challenges:

• Hyperparameter tuning: Six penalty coefficients ($\lambda_i$) balance different terms in the loss function, and their optimal setting is problem-dependent.
• Handling discontinuities: For PDEs with shocks or singularities, interface penalties may become counterproductive, as true solutions may not be smooth across subdomains.
• Sampling and computational cost: Evaluating PDE residual gradients and higher derivatives at interfaces incurs additional computational burden relative to standard PINNs.
• Scaling to high dimensions: Most results are demonstrated for 1D–2D domains; scaling to 3D or higher remains an active area for investigation.

Potential avenues for extension include adaptive interface and collocation sampling, overlapping or physics-guided domain decomposition, parallel/distributed implementation, Bayesian or multi-fidelity PINN fusion, and integration with specialized solvers for black-box hybridization.

6. Representative Applications and Broader Context

This class of PINN frameworks is applicable to a broad range of forward and inverse PDE problems in engineering and sciences, with demonstrated benefits in linear and nonlinear wave propagation, heat conduction, solid mechanics, and parametrized PDEs. The modular combination of domain decomposition and improved initialization is directly compatible with other advances in PINN theory, including higher order continuity constraints, geometric learning, physics-constrained multi-fidelity models, and efficient solvers for multiphysics systems.

References to numerical case studies and technical details can be found in (Si et al., 5 Jun 2024).