Physics-Pretrained Neural Operator

• Physics-Pretrained Neural Operators are architectures pre-trained using governing physical laws like PDEs and ODEs, offering strong data efficiency and resolution invariance.
• They incorporate high-resolution physics loss terms such as PDE residuals and boundary conditions, ensuring robust generalization across multi-scale, high-dimensional problems.
• The PPNO framework supports accurate surrogate and solver models in various fields including fluid dynamics, electromagnetics, and material science, and integrates techniques like domain decomposition and transfer learning.

A Physics-Pretrained Neural Operator (PPNO) is a neural operator architecture that is pretrained using the governing physical laws—typically in the form of partial differential equations (PDEs) or ordinary differential equations (ODEs)—rather than relying exclusively on paired input–output data. This approach leverages physics-based constraints to achieve data efficiency, strong resolution invariance, and robust generalization to complex, high-dimensional scientific problems. The PPNO paradigm subsumes standard data-driven neural operator methods, physics-informed neural operators (PINO), and recent domain decomposition strategies, and is positioned as a framework for high-fidelity surrogate and solver models in computational science and engineering.

1. Foundational Concepts

Physics-Pretrained Neural Operators are based on the operator learning paradigm, where the task is to approximate a solution operator $\mathcal{G}$ that maps input functions (e.g., coefficients, forcing terms, boundary and initial conditions) to solutions of a family of parameterized PDEs or ODEs. Unlike classical neural networks that map from finite-dimensional vectors to vectors, neural operators approximate mappings between infinite-dimensional function spaces (Li et al., 2021).

In the PPNO approach, pretraining is conducted either entirely or predominantly via physical loss terms, such as PDE residuals computed at high resolution and boundary conditions, rather than via supervised L2 error over labeled input–output pairs. This enables the model to learn the structure of the physical system before exposure to observed data, and is especially beneficial where labeled simulation data is scarce or expensive to obtain (Li et al., 2021, Chen et al., 24 Feb 2024).

The PPNO framework naturally incorporates and extends several recent advancements in neural operator research:

• Physics-informed neural operators (PINO): Combine data and physics constraints, sometimes operating without any data via physics-only pretraining (Li et al., 2021);
• Operator surrogate models: Operator components pre-trained to represent complex PDE terms (e.g., collision integrals) can be embedded as reusable modules in downstream surrogate models (Lee et al., 2022);
• Transfer and distributed learning: Pretraining across multiple related operators or domains enables robust model initialization for rapid adaptation to new physical tasks (Zhang et al., 11 Nov 2024).

2. Pretraining Strategies and Loss Formulations

Physics-pretraining involves minimizing loss functions that directly enforce the physical equations governing a system:

• PDE residual loss:

$$L_{\mathrm{PDE}}(a, u_\theta) = \int_D |\mathcal{P}(u_\theta(x), a(x))|^2 dx + \alpha \int_{\partial D} |u_\theta(x) - g(x)|^2 dx$$

where $\mathcal{P}$ is the differential operator, $a(x)$ the spatially varying coefficient (e.g., material property), $g(x)$ the prescribed boundary condition, and $u_\theta$ the neural operator’s solution prediction (Li et al., 2021).

• High-resolution residual enforcement: The PDE loss is often imposed at a higher spatial (and/or temporal) resolution than the available data, leveraging the discretization-convergent property of neural operators (such as Fourier Neural Operators) to fill in high-frequency details and achieve mesh invariance (Li et al., 2021).
• Operator surrogate losses: In cases where the system’s operator has intricate nonlocal structure (e.g., the Landau collision integral in the Fokker–Planck–Landau equation), individual components (diffusion, drift) can be learned via supervised or self-supervised proxy tasks before being incorporated into a larger physics-informed framework (Lee et al., 2022).
• Projection for invariants: For dynamical systems with conservation laws, a projection layer can be enforced as the last network block to map model outputs onto the invariant manifold defined by the first integrals (e.g., total energy or momentum), ensuring exact conservation throughout long time integration (Cardoso-Bihlo et al., 2023).

3. Architectures and Implementation Approaches

PPNOs inherit and generalize several neural operator architectures:

• Fourier Neural Operator (FNO): The backbone for many PINO and PPNO models, featuring spectral convolution layers with discretization invariance (Li et al., 2021, Ding et al., 2022).
• Deep Operator Network (DeepONet): Utilizes sensor-based branch/trunk architectures, sometimes incorporating nonlinear decoders or multi-branch strategies for high-dimensional and multi-input problems (Ramezankhani et al., 20 Jun 2024).
• Graph Neural Operators and Convolutional Neural Operators: Address nonlocality and complex, possibly heterogeneous spatial connectivity, essential for real-world physical domains (Goswami et al., 2022, Berner et al., 12 Jun 2025).
• Domain Decomposition frameworks: A single PPNO, pretrained on a canonical (simple) domain, can act as a universal local solver within a domain decomposition method, thereby enabling efficient and accurate solutions on arbitrarily complex geometries without further retraining (Wu et al., 23 Jul 2025).

A notable implementation pattern is the use of pretraining (on synthetic PDE instances or via unsupervised proxy tasks such as masked autoencoding) followed by flexible deployment or rapid task-specific fine-tuning with possibly minimal labeled data (Chen et al., 24 Feb 2024, Zhang et al., 11 Nov 2024). In PPNO-based domain decomposition, the PPNO is mapped onto each subdomain (possibly via affine transforms) and used within an additive Schwarz, iterative global assembly, or partition-of-unity framework (Wu et al., 23 Jul 2025).

4. Applications and Empirical Results

Physics-Pretrained Neural Operators have demonstrated effectiveness in a broad spectrum of computational science and engineering problems:

• Multiscale and multi-physics systems: PPNOs are capable of capturing strongly coupled nonlinear operators, for example, in geotechnical engineering, electromagnetism, quantum mechanics, fluid dynamics, and material science (Yuan et al., 26 Feb 2025).
• Large-scale and complex geometries: The L-DDM with PPNO surrogate solvers achieves strong generalization on elliptic PDEs with discontinuous microstructures, high resolution invariance, and robust out-of-distribution generalization to new unseen patterns or geometries (Wu et al., 23 Jul 2025).
• Wave propagation and scattering: Hybrid architectures such as the Waveguide Neural Operator provide high-fidelity and efficient solutions for computational electromagnetics, including challenging three-dimensional EUV mask diffraction (Es'kin et al., 5 Jul 2025).
• Long-horizon and high-frequency phenomena: Causal training, multi-step architectures, and physics-guided loss weighting in PPNOs enable accurate, stable extrapolation far beyond the training data for wave and reaction–diffusion systems (Song et al., 2 Jun 2025, Ma et al., 22 Jul 2025).
• Inverse and optimization tasks: PDE-constrained PINOs, including PPNO variants, are shown to succeed in solving inverse problems (e.g., parameter recovery) and facilitate fast optimization tasks, such as design optimization in composite processing (Ramezankhani et al., 20 Jun 2024).

Empirically, PPNO-based methods consistently outperform classical PINNs and data-driven neural operators in terms of accuracy, computational efficiency, extrapolation performance, and robustness to data sparsity (Li et al., 2021, Chen et al., 24 Feb 2024, Wu et al., 23 Jul 2025).

5. Challenges and Resolution Strategies

Several challenges addressed in PPNO research include:

• Optimization landscape: High-frequency or multi-scale physical phenomena introduce difficult loss landscapes. Operator-based parametrization and high-resolution enforcement smooth the optimization and improve convergence (Li et al., 2021).
• Computational efficiency: High-resolution physics loss is expensive; leveraging FFT-based differentiation, domain decomposition, and spectral convolution/memory-efficient kernels addresses the bottleneck (Li et al., 2021, Es'kin et al., 5 Jul 2025).
• Generalization and transferability: PPNOs can be pretrained on a wide range of synthetic or physically representative instances, supporting robust transfer via anchor or fine-tuning losses, projection layers, or LoRA-based parameter-efficient adaptation (Zhang et al., 11 Nov 2024, Cardoso-Bihlo et al., 2023).

6. Theoretical Guarantees and Future Directions

Theoretical results support the universal approximation capability and discretization convergence of PPNOs under standard regularity conditions (Li et al., 2021, Berner et al., 12 Jun 2025):

• Existence and approximation theorems: Guarantees that neural operators can approximate the true solution (operator) to within any prescribed error, provided sufficient capacity and data (Wu et al., 23 Jul 2025).
• Domain decomposition proofs: Provided the Schwarz or additive domain decomposition operators are locally Lipschitz and uniformly contractive, repeated PPNO surrogate application converges uniformly to the global PDE solution (Wu et al., 23 Jul 2025).

Future research directions encompass:

• Library development: Construction of pre-trained model libraries or "foundation operators" for rapid transfer to new physics or geometries (Li et al., 2021).
• Higher-dimensional and multi-physics coupling: Combining PPNOs with graph kernels, adaptive mesh, attention or invariant-aware neural architectures to scale to 3D and beyond (Jafarzadeh et al., 11 Jan 2024, Liu et al., 29 May 2025).
• Data/physics hybridization: Integrating self-supervised, data-driven, and physics-constrained methods (e.g., via masked autoencoders, invariant extraction) to optimize efficiency and generalizability (Zhang et al., 2023, Chen et al., 24 Feb 2024).
• Broad real-world deployment: Applying PPNOs to edge-computable and real-time engineering contexts, including structural health monitoring, digital twins, and large-scale climate or biological system modeling (Goswami et al., 2022, Ramezankhani et al., 20 Jun 2024).

7. Conclusion

Physics-Pretrained Neural Operators constitute a highly data-efficient and physically grounded strategy for science and engineering tasks that involve complex PDE systems. By leveraging governing physics for pretraining and employing architecture designs with strong inductive biases towards continuity, invariance, and generalization, PPNOs achieve superior extrapolation, computational efficiency, and adaptability to challenging, high-dimensional, multi-physics, and multi-scale settings. The framework supports a modular, reusable operator paradigm and is poised to underpin the next-generation foundation models in scientific machine learning.