Physics-Informed Neural Network Surrogates

• Physics-Informed Neural Network (PINN) surrogates are mesh-free models that integrate PDE physics into neural network training to enable rapid, high-dimensional simulations.
• They use a composite loss that balances supervised MSE errors and PDE residuals via automatic differentiation, ensuring physical consistency across varied applications.
• Architectural innovations like deep MLPs, Fourier features, and hybrid models enhance PINN performance, making them ideal for design, control, inverse problems, and uncertainty quantification.

Physics-Informed Neural Network (PINN) Surrogates are mesh-free, neural-operator-based surrogate models for partial differential equations (PDEs) that embed the governing physics in the network’s training objective. PINNs operate by minimizing a composite loss function that balances supervised errors from labeled data (snapshots, experimental measurements, or high-fidelity simulations) and unsupervised PDE residuals enforced at collocation points via automatic or numerical differentiation. Their ability to deliver parametric, high-dimensional surrogates, real-time evaluation speed, and physical consistency has led to widespread adoption for design, control, inverse problems, and uncertainty quantification across engineering, natural sciences, and computational mechanics.

1. Mathematical Formulation and Loss Structure

The core principle of PINN surrogates is the incorporation of the governing PDEs directly into the loss. For a general system

$$\mathcal{N}_{\lambda}[u](x, t; \lambda) = 0,\quad (x, t) \in \Omega \times [0, T]$$

where $\lambda$ denotes PDE parameters and boundary/initial conditions are prescribed, the PINN seeks a neural approximation $u_{\theta}(x, t; \lambda)$ with trainable parameters $\theta$, such that

$$L(\theta) = L_{\text{sup}}(\theta) + \lambda_{PDE} L_{\text{PDE}}(\theta) + L_{\text{BC}}(\theta) + L_{\text{IC}}(\theta)$$

where:

• $L_{\text{sup}}$ is a supervised loss (MSE to ground-truth or high-fidelity solvers),
• $L_{\text{PDE}}$ is the mean squared PDE residual over collocation points,
• $L_{\text{BC}}$, $L_{\text{IC}}$ enforce boundary/initial data.

For example, in river hydraulics (Zoch et al., 21 Mar 2025), the hybrid loss is: $$\mathcal{L} = \underbrace{\frac1N\sum_i|h_{\rm pred}(x_i,t_i)-h_{\rm RAS}(x_i,t_i)|^2}_{\text{sup. MSE}} + \lambda\left( \|\partial_t h + \partial_x (h u)\|^2 + \|\partial_t u + u \partial_x u + g \partial_x h\|^2 \right)$$ and in fluid dynamics surrogates (Wong et al., 2021, Ghosh et al., 2023): $\lambda$0 with $\lambda$1 encoding mass conservation, momentum conservation, and, for turbulent regimes, closure PDEs (e.g., $\lambda$2-$\lambda$3).

Physical consistency is enforced by differentiating the neural network at arbitrary points (via automatic differentiation), making the surrogate meshless and generalizable to arbitrary domain geometries or parameter values.

2. Surrogate Model Architectures

The network architecture underpinning a PINN surrogate is tailored to the complexity of the PDE, data regime, and parametric dependencies:

• Fully connected multilayer perceptrons (MLPs): Ubiquitous in PINNs; depth and width influence approximation capacity. Automated search methods (NAS-PINN (Wang et al., 2023), Auto-PINN (Wang et al., 2022)) have revealed that deeper networks do not always yield better surrogates; shallow and wide architectures can outperform very deep models for certain PDEs.
• Feature encoding: Handling high-frequency PDE response (spectral bias) often requires input lifting (e.g., random Fourier features in river stage surrogates (Zoch et al., 21 Mar 2025), sinusoidal activation initialization in CAN-PINN (Chiu et al., 2021)).
• Residual or gating blocks: Residual connections stabilize training, prevent vanishing gradients, and improve generalization (e.g., six 512-unit residual blocks in stage prediction (Zoch et al., 21 Mar 2025); gating layers in elasticity surrogates (Kag et al., 2023)).
• Output parametrization: For parametric surrogates, PDE coefficients, physical parameters, or domain IDs are concatenated to the input or processed via embeddings, enabling the model to represent solution families and deliver UQ, design, or control responses in high-dimensional parameter spaces (Panahi et al., 2024, Coulaud et al., 2024, Habich et al., 4 Feb 2025).
• Hybrid networks: FE-PINN (Sunil et al., 2024) integrates convolutional layers over FE meshes, PINN-FEM (Sobh et al., 14 Jan 2025) uses separate neural and finite-element regions—both allowing compliant meshes or strong-BC enforcement.
• Novel activations: The compleX-PINN (Si et al., 7 Feb 2025) introduces a Cauchy-inspired, learnable activation, enabling single-layer surrogates with order-of-magnitude accuracy improvement for stiff or high-dimensional PDEs.

3. Training and Optimization Strategies

PINN surrogate accuracy and robustness are sensitive to training design:

• Data sampling: Latin hypercube, Sobol quasi–Monte Carlo, or random uniform strategies ensure coverage of the $\lambda$4, $\lambda$5, and $\lambda$6 (parametric) domains for both data and collocation points (Panahi et al., 2024, Laubscher et al., 2021, Kag et al., 2023).
• Adaptive loss balancing: Composite losses are often ill-conditioned; dynamic reweighting (e.g., soft attention mechanism (Panahi et al., 2024), residual-based attention (Si et al., 7 Feb 2025), RMS-residual normalization (Ghosh et al., 2023)) is deployed to stabilize residual magnitudes and prevent trivial minimization of only a subset of terms.
• Optimizer schedules: Standard practice begins with Adam for robust, stochastic descent (typical learning rates $\lambda$7), followed by L-BFGS for full-batch quasi-Newton refinement (Laubscher et al., 2021, Sunil et al., 2024, Zubov et al., 2021).
• Transfer or stage-wise learning: Warm-start and sequential-parameter unfrozen training (as in PINN-UU (Panahi et al., 2024)) or transfer optimization for new parameter/flow regimes (Wong et al., 2021, Wang et al., 2022) accelerate convergence and leverage previously learned physics.
• Domain decomposition and mesh integration: High-fidelity surrogates for domains with complex boundaries or strong BCs employ domain splits with hybrid mesh (PINN-FEM (Sobh et al., 14 Jan 2025), FE-PINN (Sunil et al., 2024)), or physical domain decomposition (XPINN, not detailed in this selection but widely referenced).

4. Quantitative Performance, Generalization, and Applications

PINN surrogates consistently demonstrate:

• Substantial speed-ups over traditional solvers: For river hydraulics, stage surrogates achieve $\lambda$8 faster inference than HEC-RAS (Zoch et al., 21 Mar 2025); for articulated soft robots, $\lambda$9 faster than explicit Euler for real-time model predictive control (Habich et al., 4 Feb 2025); RANS-PINN achieves <1s turnaround for turbulent flow fields (Ghosh et al., 2023).
• Accurate reproduction of high-fidelity solvers: Mean relative absolute errors often $u_{\theta}(x, t; \lambda)$0, with residual physical losses on-par with discretized PDE solvers; in PINN-UU (Panahi et al., 2024), statistical moments and PDFs of outputs match $u_{\theta}(x, t; \lambda)$1-sample MC results.
• High parametric generalization: PINNs trained with parametric or domain inputs deliver real-time surrogates covering substantial ranges of uncertain, design, or environmental parameters (Panahi et al., 2024, Coulaud et al., 2024, Kag et al., 2023, Habich et al., 4 Feb 2025).
• Flexible extension to multiphysics and control: PINN surrogates have been demonstrated for coupled thermal–hydro–mechanical flows (Amini et al., 2022), dynamic elasticity and parametric UQ (Kag et al., 2023, Panahi et al., 2024), Navier–Stokes turbulence closure (Ghosh et al., 2023), and PDE-constrained optimal control (Mowlavi et al., 2021).

5. Methodological Innovations and Best Practices

Research across multiple application domains has produced key takeaways for PINN surrogate deployment:

• Fourier/positional encodings and deep input features are essential to overcome spectral bias and enable expressive surrogacy for oscillatory/steep-gradient solutions (Zoch et al., 21 Mar 2025, Chiu et al., 2021).
• Residual and mixed-variable architectures (predicting both field and auxiliary physics variables) can lower the differential order in the loss and enhance convergence in multiphysics settings (Laubscher et al., 2021, Wong et al., 2021, Sunil et al., 2024).
• Automated neural architecture search reveals optimal depth/width trade-offs and residual block placements dependent on PDE class; depth is not always superior—shallow, wide PINNs can outperform deep networks (Wang et al., 2023, Wang et al., 2022).
• Strong boundary enforcement via hybridization: When soft BC loss terms are insufficient, hybrid PINN-FEM and FE-PINN architectures enable exact essential BCs without disrupting mesh-free collocation (Sunil et al., 2024, Sobh et al., 14 Jan 2025).
• Efficient inverse and UQ workflows: PINN surrogates naturally deliver global/parametric sensitivities and uncertainty propagation via autodiff at negligible additional cost (Panahi et al., 2024); stagewise or warm-start transfer learning accelerates high-dimensional UQ and design analysis.
• Advanced activation and differentiation schemes: CAN-PINN achieves 1–2 orders of magnitude higher MSE accuracy than pure AD or ND PINNs by coupling neighbor information and AD gradients (Chiu et al., 2021). The compleX-PINN Cauchy-activated architecture provides sharp surrogate fits for stiff/high-frequency regimes (Si et al., 7 Feb 2025).

6. Limitations, Pitfalls, and Ongoing Challenges

PINN surrogate development faces several technical obstacles:

• Optimization non-convexity and local minima regularly impede convergence, requiring careful weight initialization, optimizer tuning, and loss rebalancing (Amini et al., 2022, Ghosh et al., 2023, Sunil et al., 2024).
• Loss scaling and stiffness: Multi-objective or multiphysics losses necessitate adaptive/dynamic balancing to avoid trivial or nonphysical solutions; early training phases may benefit from curriculum or gradual term introduction (Panahi et al., 2024, Ghosh et al., 2023).
• BC enforcement and mesh dependence: Standard PINNs may struggle with complex or strong Dirichlet conditions; hybridization with domain-decomposition or FE-based representations (PINN-FEM, FE-PINN) circumvents these difficulties (Sobh et al., 14 Jan 2025, Sunil et al., 2024).
• High-dimensionality and steep-gradient/oscillatory solutions: Despite meshless generalization, PINNs can experience slow convergence or reduced accuracy in high-Pe or turbulent regimes; approaches such as Fourier features, advanced activations, and operator preconditioning (e.g., Cordès scaling (Hu et al., 28 Apr 2026)) are active topics of research.
• Training and evaluation cost: While inference is orders-of-magnitude faster than classical PDE solvers, initial PINN training remains expensive and sensitive to architecture, sampling, and optimization hyperparameters (Panahi et al., 2024, Amini et al., 2022). Offline–online cost trade-offs must be considered for time-critical applications.

7. Perspectives and Future Directions

Recent advances and open questions include:

• Operator-centric architectures: Advances such as Fourier Neural Operators and domain-agnostic neural integral operators promise further scalability and mesh independence for surrogacy of complex physics (Ghosh et al., 2023).
• Automated hyperparameter/architecture optimization: Systematic neural architectural search, as in NAS-PINN and Auto-PINN, is likely to become standard for achieving robust PINN surrogates, particularly under uncertain or user-constrained computational budgets (Wang et al., 2023, Wang et al., 2022).
• Hybridization with classical solvers: PINN–FEM, FE-PINN, and related frameworks increasingly allow PINN surrogates to inherit BC compliance and mesh-adaptive properties of FEM, broadening the feasibility of surrogates in real-world engineering (Sunil et al., 2024, Sobh et al., 14 Jan 2025).
• Error control and adaptivity: Symbolic specification and error-adaptive sampling (NeuralPDE.jl (Zubov et al., 2021)) enable certified error bounds and efficient collocation in regions of high solution curvature, instrumental for scientific UQ and design.
• UQ/robustness and data-poor regimes: PINN-UU advances demonstrate statistical uncertainty quantification for PDE models, opening pathways for robust surrogate-assisted design, model validation, and active learning in experimental sciences (Panahi et al., 2024).
• Integration with design, optimization, and real-time control: The real-time deployment of PINN surrogates in MPC for robotic systems (Habich et al., 4 Feb 2025), and PDE-constrained optimal control (Mowlavi et al., 2021), signals increasing integration with control, design, and digital twin pipelines.

PINN surrogates, by fusing physical laws with the approximation power of neural networks and mesh-free parameterization, are establishing themselves as a flexible, extensible paradigm for forward, inverse, and uncertainty-aware modeling across computational science and engineering domains. Their continued evolution is driven by advances in neural architecture, mathematical theory of physics-embedding, and interfaces with classical discretization frameworks.