Physics-Informed Neural Operators

• Physics-informed neural operators are machine learning frameworks that integrate PDE residuals into training, ensuring strong physical fidelity and enhanced computational efficiency.
• They employ advanced architectures, such as layered Fourier reduction, to compress model memory and accelerate convergence across diverse parametric PDE challenges.
• These models achieve significant error reduction and efficiency improvements on benchmark problems, supporting applications in inverse design, uncertainty quantification, and control workflows.

Physics-informed neural operators (PINOs) are a class of machine learning frameworks that unify operator learning with direct enforcement of governing physical laws via the training objective. Distinct from conventional supervised neural operators—such as DeepONet or the Fourier Neural Operator (FNO)—which are trained predominantly on labeled input–output function pairs, PINOs incorporate residuals of partial differential equations (PDEs) and physical constraints into either the sole or dominant component of the loss. This leads to models capable of learning parametric solution operators that generalize to new PDE inputs, often from limited or even zero paired data, with strong physical fidelity and computational efficiency (Li et al., 2021, Wang et al., 21 Jun 2025, Zhang et al., 6 Nov 2025).

1. Mathematical Foundations and Problem Setting

Physics-informed neural operators formalize the learning of infinite-dimensional solution operators for broad families of PDEs. The general setting is a parametric PDE system: $$\begin{cases} \mathcal{R}_x^{\eta_{\mathrm r}}[u](x,t) = 0 & (x,t)\in\Omega\times[0,T], \ \mathcal{B}_x^{\eta_{\mathrm{bc}}}[u](x,t) = 0 & (x,t)\in\partial\Omega\times[0,T], \ \mathcal{I}_x^{\eta_{\mathrm{ic}}}[u](x,0) = 0 & x\in\Omega, \end{cases}$$ where $$\eta = (\eta_{\mathrm{r}}, \eta_{\mathrm{bc}}, \eta_{\mathrm{ic}})$$ parameterizes the family. The solution operator $\mathcal{G}: \mathcal{H} \to \mathcal{U}$ maps PDE parameters $\eta \in \mathcal{H}$ to functions $u(\cdot;\eta)$ in a function space $\mathcal{U}$ (Wang et al., 21 Jun 2025, Zhang et al., 6 Nov 2025).

PINOs aim to approximate $\mathcal{G}$ by a neural operator $f(x,\eta; \theta)$, trained such that for each sampled $\eta$, $f$ yields $u_\eta(x)$ satisfying the PDE and its boundary/initial conditions in a weak sense. The physics-informed loss, constructed as

$$L_\eta[u] = \| \mathcal{R}_x^{\eta_{\mathrm r}}[u] \|^2_{L^2(\Omega \times [0,T])} + \lambda_{\mathrm{bc}} \| \mathcal{B}_x^{\eta_{\mathrm{bc}}}[u] \|^2_{L^2(\partial\Omega \times [0,T])} + \lambda_{\mathrm{ic}} \| \mathcal{I}_x^{\eta_{\mathrm{ic}}}[u] \|^2_{L^2(\Omega)},$$

anchors the model to the PDE solution manifold and is typically approximated via Monte Carlo sampling over collocation points (Wang et al., 21 Jun 2025, Li et al., 2021). Optionally, a data loss term can be included when paired target data are available, yielding a composite loss.

2. Representative Architectures and Frequency-Domain Compression

PINOs admit diverse architectural realizations, including operator-valued neural networks parameterized by the PDE parameters and spatial/temporal coordinates, with physics constraints imposed via automatic or finite difference–based derivatives. A notable development is the Layered Fourier Reduced PINO (LFR-PINO), which introduces two principal innovations (Wang et al., 21 Jun 2025):

• Layered hypernetwork architecture: Instead of a monolithic hypernetwork mapping parameters $\eta$ to all weights, LFR-PINO employs $L$ separate hypernetworks $F_1, \dots, F_L$, each producing the weights for one layer of the main $L$-layer network. Each $F_l$ outputs only the truncated set of low-frequency Fourier coefficients $\widehat{W}^{(l)}$ for that layer's flattened weight vector.
• Frequency-domain reduction: By retaining only the lowest $p$-frequency modes ($p \ll N$) for each layer's weight vector (where $N$ is the weight dimensionality), the memory and computational footprint are reduced by a factor of $N/p$. The weights in the physical space are recovered via an inverse DFT, exploiting the bias of neural networks towards low-frequency content (the "frequency principle"):

$$W^{(l)}_j(\eta) = \frac{1}{N} \sum_{k=0}^{p-1} \widehat{W}^{(l)}_k(\eta)\,e^{2\pi i k j/N}.$$

This design minimizes redundancy in the parameter-to-weight maps and significantly compresses model memory usage (by 28.6–69.3% relative to monolithic hypernetworks) without degrading accuracy (Wang et al., 21 Jun 2025).

3. Training Methodologies: Physics-Informed Losses and Fine-Tuning

Training PINOs is characteristically distinct from data-driven neural operators:

• Pre-training: The hypernetwork parameters $\theta$ are optimized over a set of PDE instances (varying $\eta$), using the physics-informed loss over a set of collocation points. For $M$ sampled parameters, the objective is:

$$\theta^* = \arg\min_\theta \sum_{i=1}^M \hat L_{\eta^{(i)}} [f(\cdot, \eta^{(i)}; \theta)],$$

with each $\hat L_{\eta^{(i)}}$ evaluated as described above.

• Fine-tuning: For a new, unseen parameter $\eta_{\text{new}}$, the hypernetwork can be further tuned by minimizing $\hat L_{\eta_{\mathrm{new}}}$, either over all parameters or selected submodules (e.g., only the final layer hypernet), achieving fast adaptation with few steps (Wang et al., 21 Jun 2025).

For practical implementation, empirical studies sample collocation points for residual, boundary, and initial conditions using pseudo-random quadrature or uniform gridding. The method scales efficiently as the frequency-domain reduction ensures that the size and cost of the parameter-to-weight mapping is $O(Lp)$, a reduction by up to a factor of $N/p$ compared to monolithic designs.

4. Accuracy, Efficiency, and Benchmarking

LFR-PINO and related PINO models have demonstrated state-of-the-art accuracy and efficiency across diverse parametric PDE benchmarks:

Method	Anti-derivative	Advection	Burgers’	Diff-React
PI-DeepONet	$3.82\!\times\!10^{-3}$	$4.97\!\times\!10^{-2}$	$4.54\!\times\!10^{-2}$	$8.23\!\times\!10^{-2}$
MAD	$2.15\!\times\!10^{-2}$	$3.36\!\times\!10^{-2}$	$1.59\!\times\!10^{-1}$	$1.79\!\times\!10^{-1}$
Hyper-PINNs	$4.86\!\times\!10^{-3}$	$1.98\!\times\!10^{-2}$	$4.45\!\times\!10^{-2}$	$6.56\!\times\!10^{-2}$
LFR-PINO	$\mathbf{3.36\!\times\!10^{-3}}$	$\mathbf{6.21\!\times\!10^{-3}}$	$\mathbf{3.93\!\times\!10^{-2}}$	$\mathbf{3.92\!\times\!10^{-2}}$

LFR-PINO achieves a 22.8%–68.7% reduction in relative L2 error compared to state-of-the-art baselines, and up to ~70% reduction in memory footprint (Wang et al., 21 Jun 2025). The frequency-domain reduction improves convergence rate and enhances optimization stability. Performance is preserved across various PDE types (e.g., advection, anti-derivative, nonlinear Burgers', diffusion-reaction systems) and for parameter draws spanning the input space.

5. Model Efficiency and Computational Scaling

The layered Fourier-reduced approach enables near-linear scaling of memory and computation in the number of network layers and preserved spectral modes. For a main network with width $n$:

• Monolithic "Hyper-PINN" scales as $O(n^2)$ in parameter mapping.
• LFR-PINO, with $p/N \sim 10$–$25\%$, brings an $O(Lp)$ scaling, compressing the weight mapping by a factor $N/p$.

Measured across four representative PDE problems, LFR-PINO reduces memory usage from, e.g., 10.44 MB to 3.21 MB, and—due to effective low-rank recovery of weights—does not compromise solution fidelity (Wang et al., 21 Jun 2025).

A modest runtime penalty for decentralized hypernetwork evaluation is observed, but total inference cost remains suitable for real-time applications, including multi-query design and control workflows.

6. Theoretical Significance and Limitations

The spectral compression rationale is supported by the empirical “frequency principle” in deep networks: low-frequency weight subspaces dominate function expressivity in many physics-driven tasks. Truncating high-frequency modes in the weight parameterization thus preserves solution accuracy and enhances computational tractability.

Potential limitations of the approach stem from its restriction to cases where Fourier bases (or a similar global basis) provide an efficient description of the network's function space. Highly localized or singular solutions may require extension to alternative kernel representations or hybridization with non-spectral bases.

7. Outlook: Generalizability and Future Directions

LFR-PINO exemplifies the convergence of operator-learning, hypernetwork modularization, and physics-informed training for PDE surrogate modeling. Its achievements in balancing solution fidelity, memory savings, and adaptability have particular relevance for fields requiring universal and reusable solvers—such as aerospace inverse design, uncertainty quantification, and parametric studies.

Future research will likely extend the frequency-domain reduction and layered parameterization to learnable basis selection, anisotropic spectral compression (e.g., wavelet or multi-scale frameworks), adaptive layer-wise hypernetwork complexity, and hybridization with data-driven fine-tuning for out-of-distribution robustness (Wang et al., 21 Jun 2025, Li et al., 2021, Zhang et al., 6 Nov 2025).

---

References

• LFR-PINO: Layered Fourier Reduced Physics-Informed Neural Operator for Parametric PDEs (Wang et al., 21 Jun 2025)
• Physics-Informed Neural Operator for Learning Partial Differential Equations (Li et al., 2021)
• Physics-Informed Neural Networks and Neural Operators for Parametric PDEs: A Human-AI Collaborative Analysis (Zhang et al., 6 Nov 2025)