Physically Constrained Neural Operators

• Physically constrained neural operators are neural network-based models that embed physical laws like boundary conditions, conservation laws, and symmetries to ensure well-posed PDE solutions.
• They employ techniques such as hard boundary enforcement, physics-informed loss functions, and invariant kernel designs to mitigate violations common to standard neural operators.
• Empirical evidence demonstrates that these strategies achieve 2–30× error reduction while enhancing data efficiency and preserving theoretical guarantees in complex simulations.

Physically constrained neural operators are neural network-based models for operator learning that integrate fundamental physical constraints—such as boundary conditions, conservation laws, symmetries, or internal variational structure—directly into their architecture, parameterization, or training regime. These constraints arise from underlying physical laws expressed as PDEs and guarantee the physical admissibility, well-posedness, and generalizability of learned solution operators. This paradigm aims to resolve deficiencies in standard data-driven neural operators, which may fit training data well but violate essential constraints—such as BCs or conservation laws—on out-of-distribution or boundary domain points.

1. Operator Learning and the Necessity of Physical Constraints

A neural operator learns a map $T : a(x) \mapsto u(x,t)$, where $a(x)$ represents input functions (e.g., boundary data, initial field, source terms), and $u(x,t)$ is the solution of a PDE or spatio-temporal evolution problem. In its prototypical form, $T$ is expressed as a linear or nonlinear integral operator:

$u(x,t) = (T a)(x) = \int_\Omega K(x,y) a(y) \, dy,$

with kernel $K(x,y)$ parameterized (typically by neural networks) and learned from data. However, in physical systems, $u(x,t)$ is required to satisfy strict constraints:

• Boundary conditions (Dirichlet, Neumann, periodic, etc.) for well-posedness,
• Conservation laws (mass, energy, momentum) for physical fidelity,
• Symmetries (translation, rotation invariance/equivariance) for correct invariants,
• Internal constitutive structure (e.g., flux–gradient relations).

Empirical evidence indicates that unconstrained neural operators (e.g., FNOs, DeepONets) frequently incur significant violations at the domain boundary or under transformations, undermining physical reliability (Saad et al., 2022, Liu et al., 2023, Liu et al., 2022).

2. Architectural and Training Strategies for Constraint Enforcement

Physically constrained neural operators employ a hierarchy of strategies for integrating constraints:

• Boundary Condition Enforcement ("hard BCs"): The Boundary enforcing Operator Network (BOON) (Saad et al., 2022) introduces a post-processing correction of the kernel matrix in Fourier or integral-operator layers. For discrete grid $x_i$:
  • Dirichlet: Replace the boundary row $K_{bdy}[i_b, \cdot]$ such that $\tilde{u}(x_{i_b}) = \alpha_D(x_{i_b}, t)$ exactly.
  • Neumann: Modify rows via finite-difference stencils to match prescribed normal derivatives.
  • Periodic: Symmetrize boundary rows: $K_{bdy}[0, \cdot] = K_{bdy}[N-1, \cdot]$.
  • The refinement algorithms require only a small number of kernel applications and produce zero empirical boundary error.
• Physics-Informed Loss ("soft constraints"): Physics-Informed Neural Operators (PINO) (Azizzadenesheli et al., 2023, Goswami et al., 2022) incorporate PDE residuals, boundary penalties, and initial-condition penalties into the composite loss:

$$\mathcal{L} = \mathcal{L}_{\text{data}} + \lambda_{PDE} \mathcal{L}_{PDE} + \lambda_{bc} \mathcal{L}_{bc} + \lambda_{reg} \mathcal{L}_{reg},$$

where physical constraints are imposed at interior and boundary collocation points using automatic differentiation.

• Conservation Law Encoding (e.g., divergence-free outputs): Conservation law-encoded neural operators (clawNOs) (Liu et al., 2023) guarantee $\nabla \cdot u = 0$ by parameterizing $u$ as the divergence of a learned skew-symmetric potential via differential forms,

$$u_j(x) = \sum_{k=1}^p \partial_{x_k} \mu_{jk}(x), \quad \mu_{jk} = -\mu_{kj},$$

imposing mass or volume conservation exactly by architecture.

• Symmetry-Invariant Kernels: Invariant Neural Operators (INO) (Liu et al., 2022) replace kernel arguments with frame-invariant edge features (distance, local orientation) and restrict pointwise features to rotation-invariant scalars, ensuring translation/rotation invariance and—by Noether's theorem—guaranteed conservation of linear and angular momentum.
• Internal Variable and Constitutive Structure: Physically-Guided Neural Networks with Internal Variables (PGNNIV) (Ayensa-Jiménez et al., 2020) embed balance laws and constitutive relations directly into the network topology (e.g., convolutional layers realize finite-difference operators, internal neurons track physical states), penalizing residuals as part of the loss.
• Linearly Constrained Output Spaces: Linearly Constrained Neural Networks (Hendriks et al., 2020) enforce $C[f] = 0$ (e.g., $\nabla \cdot f = 0$) by parametrizing $f = G[g]$ for a potential $g$, such that the constraint holds for any $g$ by construction.
• Sensitivity-based Regularization: Sensitivity-Constrained FNO (SC-FNO) (Behroozi et al., 13 May 2025) extends FNOs to match not only the solution path but also $\partial u/\partial p$ with respect to parameters, enforcing physical consistency for forward/inverse tasks and improving robustness to parameter drift.

3. Theoretical Guarantees and Limitations

Several physically constrained neural operator strategies admit explicit theoretical guarantees:

• Existence and Eliminability: For any learned kernel $K$, there exist row operations yielding a corrected $K_{bdy}$ so that the discrete operator satisfies Dirichlet, Neumann, or periodic BCs on the grid; the boundary error is eliminated exactly (Saad et al., 2022).
• Boundedness: The $L^2$ difference between the vanilla operator and its boundary-corrected analogue is controlled in terms of the original boundary residual; tighter bounds are available for periodic corrections.
• Universal Approximation: Physics-informed and invariant operator architectures retain the universal approximation property for functionals on Sobolev spaces, with discretization-convergence holding under mesh refinement (Azizzadenesheli et al., 2023, Liu et al., 2022).
• Noether-Type Conservation: INO's form-invariant kernels ensure that operator solutions preserve conservation laws under domain translation and rotation, with formal theorems verifying equivariance (Liu et al., 2022).
• Divergence-free Fidelity: For clawNOs, the divergence-free constraint is satisfied up to the numerical error of the differentiation sublayer; explicit error rates for spectral/meshfree discretization are provided (Liu et al., 2023).

However, several limitations persist:

• Most frameworks focus on linear constraints (Dirichlet, Neumann, periodic, conservation laws). Extensions to nonlinear or nonlocal constraints (e.g., Robin BCs, moving boundaries, coupled multiphysics, or high-order interface constraints) remain under active development.
• Geometric complexity (e.g., curvilinear or highly irregular boundaries) may require sophisticated local discretizations for derivative approximations.
• Most "hard" methods do not penalize the PDE residual within $\Omega$; hybrid schemes with physics-informed losses are necessary for full interior control.
• Resource footprint grows linearly with O(N) added memory or operator calls, but remains modest due to the independence from the underlying grid resolution.

4. Empirical Performance Across Canonical Problems

The impact of physical constraints is substantiated by numerical experiments across various PDE benchmarks:

Model / Constraint	Problem	Relative $L^2$ Error	Boundary / Constraint Error	Improvement Factor
FNO (vanilla)	Burgers (Dirichlet)	$2.8 \times 10^{-3}$	nonzero	—
BOON (hard-BC)	Burgers (Dirichlet)	$8.4 \times 10^{-5}$	$0$	$30\times$
FNO	Heat (Neumann)	$1.24 \times 10^{-2}$	$O(6.3)$	—
BOON	Heat (Neumann)	$1.19 \times 10^{-2}$	$0$	$\gg 2\times$
SC-FNO	Burgers, $82$ params	$0.0073$	$-$	$4\times$
INO	Darcy flow	$1.2\%$	inv. under trans/rot	$2\times$ vs. GNO
clawFNO	NS, 2D incompressible	$1.2\%$ (small N)	$\nabla \cdot u = 0$ (exact)	$2\times$ vs. FNO

Physical constraint enforcement yields 2–30$\times$ improvements in solution error and drives empirical constraint violation to machine precision, even in small-data regimes. Data efficiency is significantly enhanced: constrained models achieve target error rates with 2–10$\times$ fewer training examples or model parameters relative to unconstrained or penalty-based approaches (Saad et al., 2022, Behroozi et al., 13 May 2025, Liu et al., 2023).

5. Practical Implementation Workflow

The following procedural elements are critical for practical deployment of physically constrained neural operators:

1. Model Selection: Choose an operator architecture aligned with the domain (FNO for regular grids/global coupling; GNO/Geo-FNO for irregular/complex domains; DeepONet for sensorized inputs).
2. Constraint Integration: Incorporate constraints architecturally (BOON, INO, clawNOs, linear constraints) or as loss penalties (PINO, PGNNIV, sensitivity penalties).
3. Collocation and Differentiation: Determine collocation points for interior and boundary enforcement; use automatic differentiation or spectral/finite-difference methods for gradient computations as required.
4. Hyperparameter Tuning: Balance data, PDE, and constraint losses (penalty weights) via cross-validation.
5. Hybridization and Fine-Tuning: Combine "hard" architectural constraints with "soft" interior penalties for robust enforcement; allow instance-specific fine-tuning via pure PDE loss.
6. Inference and Design: Utilize the differentiability of the learned operator for downstream tasks, including inverse design, parameter inference, or uncertainty quantification.

The architectural corrections are typically lightweight, incur negligible overhead (O(N) memory, 2–3 kernel calls), and preserve the mesh/parameter-independence of the backbone neural operator.

6. Types of Physical Constraints and Extensibility

The principal physical constraints incorporated to date include:

• Boundary Conditions: Hard imposition of Dirichlet, Neumann, and periodic boundaries via kernel correction; Robin BCs require further theoretical extensions.
• Conservation Laws: Directly encoded via divergence-free or curl-free output layers or via potential function parameterizations.
• Symmetry Enforcement: Invariances (translation, rotation) embedded in the kernel argument structure, guaranteeing conservation laws via Noether's theorem.
• Constitutive Modeling: Integration of state equations and balance laws in network layers; neuron-level interpretation as physical state variables.
• Sensitivity/Parameter Consistency: Explicit matching of solution derivatives with respect to parameters, critical for robust inverse modeling and concept-drift resilience.

Current methods are most mature for single-constraint enforcement in single-physics scenarios. Open research directions include generalization to multi-physics, nonlocal interface constraints, higher-order differential constraints, and simultaneous enforcement of multiple conservation laws (Liu et al., 2023).

7. Role in Scientific Computing and Future Directions

Physically constrained neural operators represent a paradigm shift in surrogate modeling for scientific computing. By design, these models yield predictions that are guaranteed to honor the most important physical requirements—ensuring not only accuracy but also consistency, robustness, and interpretability. As modern simulation demands escalate in high-dimensional, heterogeneous, and multiphysics settings, physically constrained operator learning is positioned as the unifying methodology bridging classic numerical analysis and scalable, data-driven deep learning (Saad et al., 2022, Azizzadenesheli et al., 2023, Liu et al., 2023).

Key ongoing challenges encompass:

• Extension to nonlinear, multiphysics, and moving boundary problems,
• Systematic methods for generic constraint encoding (beyond linear BCs and conservation),
• Integration with uncertainty quantification and design/optimization pipelines,
• Theoretical analysis of capacity, sample complexity, and transferability in constrained operator families.

These directions will require continued collaboration between numerical analysis, applied mathematics, and machine learning communities to forge high-fidelity, explainable, and universally robust neural surrogates for complex physical systems.