Physics-Constrained Neural Networks (PECANN)

• Physics-Constrained Neural Networks are models that embed physical laws directly into their architecture to strictly enforce governing equations and boundary conditions.
• They utilize adaptive augmented Lagrangian methods and per-constraint penalty updates to minimize errors and improve training stability.
• PECANN achieve significantly lower error rates and enhanced computational efficiency in forward/inverse PDEs, fluid dynamics, and multiscale applications.

Physics-Constrained Neural Networks (PECANN)

Physics-Constrained Neural Networks (PECANN) are a class of machine learning models that integrate physical laws—such as conservation, symmetries, and governing equations—directly into the neural network architecture or training objective. This approach sharply narrows the hypothesis space to physically admissible functions, yielding surrogates with greater interpretability, higher data efficiency, and improved accuracy compared to unconstrained or purely penalty-driven networks. PECANN distinguishes itself by enforcing physics through hard equality constraints, adaptive optimization schemes, or architectural embedding, rather than solely penalizing violations in the loss function.

1. Mathematical Foundations and Core Formulation

The canonical PECANN framework formulates neural PDE modeling as a constrained optimization problem, where the neural network $u_\theta(x,t)$ must satisfy a system of equality constraints corresponding to the governing equations and boundary/initial conditions. These constraints are typically written as

$$\begin{aligned} &R\big(u_\theta; x, t\big) = 0 \quad \text{(PDE residual at interior/observation points)} \ &B\big(u_\theta; x, t\big) = 0 \quad \text{(boundary/initial conditions or high-fidelity data)} \end{aligned}$$

Optionally, the objective incorporates data fitting or physical regularization terms (e.g., likelihood terms for noisy/multifidelity data).

To enforce these constraints, PECANN employs augmented Lagrangian methods (ALM), which blend the merits of classical penalty and Lagrange multiplier-based optimization. The network minimizes the augmented Lagrangian: $$L_{\text{ALM}}(\theta, \lambda; \mu) = J_F(\theta) + J_D(\theta) + \sum_{i} \lambda_{R}^{(i)}\,R(u_\theta; x_i, t_i) + \sum_{k} \lambda_{B}^{(k)}\,B(u_\theta; x_k, t_k) + \frac{\mu}{2}\bigg(\sum_{i}\lvert R(u_\theta; x_i, t_i) \rvert^2 + \sum_{k}\lvert B(u_\theta; x_k, t_k) \rvert^2\bigg)$$ where $J_F$ and $J_D$ are, respectively, PDE-residual and data-matching objectives. The optimization alternates between primal neural parameter updates and dual multiplier updates, with penalties increased conditionally when constraint violation plateaus (Basir et al., 2021).

2. Constraint Enforcement Strategies and Algorithmic Advances

2.1 Adaptive Augmented Lagrangian Approaches

Direct enforcement across heterogeneous constraints (PDE, boundary, data) can be numerically stiff due to scale and conditioning disparities. Recent extensions decouple penalties, assigning each constraint its own adaptively updated penalty parameter $\mu_i$, akin to per-constraint RMSProp. These are updated according to the RMS of residuals: $$\bar v_i^{\,t} = \alpha\,\bar v_i^{\,t-1} + (1-\alpha)\,C_i(\theta^t)^2,\quad \mu_i = \frac{\gamma}{\sqrt{\bar v_i + \epsilon}}$$ with Lagrange multipliers advanced via $$\lambda_i^{\,t} = \lambda_i^{\,t-1} + \mu_i\,C_i(\theta^t)$$ (Basir et al., 2023, Hu et al., 21 Aug 2025). This conditionally adaptive penalty update (CAPU) accelerates convergence, improves numerical stability, and allows mini-batch implementations suitable for large-scale or streaming scenarios.

2.2 Expectation-based and Domain Decomposition Formulations

Aggregating constraints over sampled collocation points ($\mathbb{E}[c_i] = 0$) further reduces memory and enhances mini-batch stochastic training efficiency (Basir et al., 2023, Hu et al., 21 Aug 2025). For computational scalability on large domains or high-wavenumber/multiscale problems, non-overlapping Schwarz-type domain decomposition schemes have been integrated with PECANN: independent subnetworks are deployed over subdomains, with interface conditions enforced by generalized transmission operators and local augmented Lagrangians. Communication is limited to periodic exchange of boundary values, ensuring efficient parallelization (Hu et al., 2024).

3. Architectures, Loss Functions, and Physical Law Encoding

PECANN architecture typically employs feedforward MLPs or shallow residual networks with tanh activations to parameterize the solution field $u_\theta(x,t)$ (Basir et al., 2021). Physics is enforced by inclusion of physical constraint residuals—PDE operators, boundary mismatches, and high-fidelity data—either as hard constraints in the optimizer or as non-trainable normalization layers and architectural elements.

For time-evolving PDEs or inverse problems, PECANN leverages time-windowing to partition long intervals into tractable sub-problems, with window-to-window continuity enforced via initial condition constraints in the ALM objective (Hu et al., 21 Aug 2025). For systems with oscillatory or multi-scale features, a Fourier feature mapping is applied to the inputs, greatly enhancing the representational capacity for capturing high-frequency modes without resorting to deep MLPs (Hu et al., 21 Aug 2025, Hu et al., 24 Nov 2025).

In cases requiring exact preservation of physical invariants, architectural design can further enforce conservation laws by construction (e.g., divergence-free outputs via potential fields, unitarity via parameterization) (Hendriks et al., 2020, Jnini et al., 2 Mar 2025). Recent developments also include neural projection layers for strict satisfaction of algebraic/differential constraints (DAE-HardNet), which iteratively project network predictions onto the constraint manifold using an inner Newton–KKT solve (Golder et al., 5 Dec 2025).

4. Practical Performance and Empirical Results

PECANN methods demonstrate orders-of-magnitude lower error and tighter constraint satisfaction compared to traditional PINNs or penalty-driven models across diverse problem classes:

• Forward/Inverse PDEs: For Poisson, Helmholtz, and reaction–diffusion equations, PECANN achieves relative $$\begin{aligned} &R\big(u_\theta; x, t\big) = 0 \quad \text{(PDE residual at interior/observation points)} \ &B\big(u_\theta; x, t\big) = 0 \quad \text{(boundary/initial conditions or high-fidelity data)} \end{aligned}$$0 errors of $$\begin{aligned} &R\big(u_\theta; x, t\big) = 0 \quad \text{(PDE residual at interior/observation points)} \ &B\big(u_\theta; x, t\big) = 0 \quad \text{(boundary/initial conditions or high-fidelity data)} \end{aligned}$$1, 100–1,000× lower than PINNs, robust both on irregular domains and in the presence of noisy/multifidelity data (Basir et al., 2021, Basir et al., 2023).
• Navier–Stokes Flows: Fourier-featured PECANN surrogates resolve high-Reynolds-number cavity and open flows up to $$\begin{aligned} &R\big(u_\theta; x, t\big) = 0 \quad \text{(PDE residual at interior/observation points)} \ &B\big(u_\theta; x, t\big) = 0 \quad \text{(boundary/initial conditions or high-fidelity data)} \end{aligned}$$2, attaining boundary/divergence residuals below $$\begin{aligned} &R\big(u_\theta; x, t\big) = 0 \quad \text{(PDE residual at interior/observation points)} \ &B\big(u_\theta; x, t\big) = 0 \quad \text{(boundary/initial conditions or high-fidelity data)} \end{aligned}$$3, and matching key hydrodynamic metrics to within 1% of benchmarks (Hu et al., 24 Nov 2025).
• High-Wavenumber/Multiscale Problems: In large-domain Helmholtz or multi-phase flow models, Schwarz-type PECANN domain decomposition yields systematic error reduction as subdomain count increases, maintaining $$\begin{aligned} &R\big(u_\theta; x, t\big) = 0 \quad \text{(PDE residual at interior/observation points)} \ &B\big(u_\theta; x, t\big) = 0 \quad \text{(boundary/initial conditions or high-fidelity data)} \end{aligned}$$4 errors down to $$\begin{aligned} &R\big(u_\theta; x, t\big) = 0 \quad \text{(PDE residual at interior/observation points)} \ &B\big(u_\theta; x, t\big) = 0 \quad \text{(boundary/initial conditions or high-fidelity data)} \end{aligned}$$5 (Hu et al., 2024, Zheng et al., 2021).
• Conservation Laws and Inverse Inference: Architectures enforcing mass, momentum, or energy conservation by design (e.g., MCBOM for multiphase flows, Riemann Tensor NNs for continuum systems) achieve constraint satisfaction to machine precision, preventing drift and reducing required training data by an order of magnitude (Jnini et al., 2 Mar 2025, Zheng et al., 2021).

5. Extensions, Specializations, and Related Methodologies

PECANN methodology is extensible to a spectrum of physical systems and modeling scenarios:

• Multi-fidelity and Data Fusion: High-fidelity (equality) and low-fidelity (likelihood) data can be seamlessly integrated, automatically re-balanced by ALM without manual weighting (Basir et al., 2021, Hu et al., 21 Aug 2025).
• Operator Learning and Surrogate Design: The ALM-based training setup can be paired with advanced surrogate frameworks (e.g., Fourier or LordNet operators) or composed with spatial-temporal decomposition for ultra-fast PDE surrogates (NeuralStagger), achieving 10–100× acceleration (Huang et al., 2023).
• Design and Inverse Mapping: "Physics-constrained" inverse designs leverage differentiable forward-physics modules (embedded surrogates) to rapidly infer input geometries/materials matching target outputs, with physics-based loss ensuring functional feasibility (Feng et al., 2022).
• Architectural Constraints: Direct encoding of symmetry, even-odd structure, and conservation laws as invariance layers or constraint-enforcing heads further shrinks the function class under consideration, enhancing noise robustness and symbolic interpretability (Barber et al., 2021).

6. Limitations, Open Problems, and Future Directions

Despite substantial advances, open questions remain:

• Inequality and Unilateral Constraints: Most PECANN realizations enforce hard equalities; embedding robust support for inequality or unilateral physics constraints remains challenging.
• Stiff or Multiscale Systems: Extremely stiff systems, or those with disparate scales, may still require specialized feature maps or decomposition strategies to maintain spectral accuracy.
• Optimization and Scalability: ALM and adaptive penalty schemes require judicious hyperparameter selection; research continues into automated update strategies, efficient second-order optimization, and handling of ill-conditioned constraint systems (Basir et al., 2023, Hu et al., 21 Aug 2025).
• Projection Layer Integration: Direct manifold projection (e.g., DAE-HardNet) guarantees strict constraint satisfaction but increases computational load; efficient, hybrid projection-penalty architectures are an active area (Golder et al., 5 Dec 2025).

Continued development targets robust meshless solvers for nonlinear/multi-physics PDEs, operator-valued surrogates for time-dependent and parameterized problems, and integration with experimental data for uncertainty-quantified scientific machine learning.

---

Key References:

• PECANN and adaptive ALM: (Basir et al., 2021, Basir et al., 2023, Hu et al., 21 Aug 2025)
• Domain decomposition and high-wavenumber Helmholtz: (Hu et al., 2024)
• Conservation-by-architecture (e.g., Riemann Tensor NNs): (Jnini et al., 2 Mar 2025)
• Projection-layer/hard-constraint formulation: (Golder et al., 5 Dec 2025)
• Multiphase and material design applications: (Zheng et al., 2021, Feng et al., 2022)