Deep Neural Operator Framework

• Deep neural-operator frameworks are computational methods that learn mappings between infinite-dimensional function spaces, enabling mesh-invariant surrogate models.
• They employ branch and trunk network architectures, such as DeepONet and FNO, to decompose and approximate complex nonlinear operators efficiently.
• Applications in PDE solving, inverse problems, and digital twins demonstrate high accuracy with errors as low as 0.33% in challenging computational scenarios.

A deep neural-operator (DNO) framework is a computational paradigm for learning mappings between function spaces—i.e., nonlinear operators—via specially designed neural network architectures that generalize standard deep learning from finite-dimensional regression to the approximation of input–output maps between infinite-dimensional Banach spaces. DNOs enable rapid, mesh-agnostic, and generalizable inference for nonlinear models in computational science, with applications ranging from solution surrogates for families of partial differential equations (PDEs), optimization of complex dynamical systems, uncertainty quantification, inverse problems, and full-scale digital twins.

1. Mathematical Foundation and Universal Approximation

DNOs formalize the regression problem as learning an operator $G : V \to U$, where $V$ and $U$ are (infinite-dimensional) spaces of real- or vector-valued functions, such as spaces of input coefficient fields, initial/boundary conditions, or forcing terms, and the corresponding spaces of solution fields. The key theoretical underpinning is the operator universal approximation theorem (Goswami et al., 2022), which states that, for any continuous nonlinear operator $G$ on compact metric subspaces of $C(K)$, there exists a DNO—typically realized as a composition of two neural networks—that can approximate $G$ uniformly to any prescribed accuracy:

$G(f)(y) \approx \sum_{k=1}^p b_k(f) t_k(y),$

where $b_k$ and $t_k$ are neural networks (the “branch” and “trunk” nets, respectively), and $p$ is the size of the latent feature space.

The operator learning task is thus to construct and train neural architectures that realize such decompositions, both for resolution-invariant learning and for complex, possibly multi-physics, and multi-output problems.

2. Canonical Architectures: DeepONet and Variants

Deep Operator Network (DeepONet)

DeepONet implements the operator ansatz via two neural subnetworks:

• The branch net $b : \mathbb{R}^m \to \mathbb{R}^p$ encodes a discretization of the input function $f$ at $m$ sensor points $\{\eta_j\}$.
• The trunk net $t : \mathbb{R}^{d_y} \to \mathbb{R}^p$ encodes the query coordinate $y$ (spatial, temporal, or spatio-temporal).
• The predicted output is

$$G_{\theta}(f)(y) = \langle b(f), t(y) \rangle = \sum_{k=1}^p b_k(f)t_k(y),$$

where $\theta$ collects all trainable parameters.

This framework supports universal approximation, is mesh-invariant (outputs can be evaluated at arbitrary query points), and accommodates multiple outputs by concatenating or replicating the trunk branch (Goswami et al., 2022).

Fourier Neural Operator (FNO) and Others

For translation-invariant (e.g., periodic) problems, FNO introduces global Fourier convolution layers acting in the latent space, enabling efficient global coupling, resolution independence, and linear-scaling computation with respect to the mesh size (Goswami et al., 2022).

Extensions include Graph Neural Operators and Nonlocal Kernel Networks, which utilize graph-based or integral-kernel representations for learning solution maps on unstructured domains or nonlocal operators.

3. Physics-Informed Operator Learning

DNOs admit “purely data-driven” or “physics-informed” (PINN-style) loss formulations. When the governing equations are known, the loss function is augmented with PDE-based penalties:

$$L(\theta) = L_{\text{data}}(\theta) + L_{\text{phys}}(\theta),$$

where $L_{\text{data}}$ encodes empirical errors, and the physics-based term decomposes as

$$L_{\text{phys}} = L_{\text{ic}} + L_{\text{bc}} + L_{\text{res}},$$

with initial, boundary, and interior (residual) contributions. Residuals are evaluated by automatic (or Fourier) differentiation through the DNO outputs (Goswami et al., 2022). This enables data-efficient learning, regularization in the presence of scarce or noisy measurements, and enforcement of physical constraints—valuable for scientific and engineering inverse problems (Kag et al., 6 Dec 2024).

4. Advanced Specializations and Algorithmic Extensions

Recent methodological advances have expanded DNOs' flexibility:

• Feature expansions: Inclusion of explicit physical features (e.g., POD modes, trigonometric time bases, autoregressive lags) as either part of the trunk or the branch, accelerating convergence and improving extrapolation.
• Multiple-input neural operators (MIONet, MONet, MNO): Simultaneously encode several input function fields (e.g., spatially varying coefficients and initial states), structuring the operator as

$$G(v, w)(y) = \sum_{k=1}^p b_k^v(v)\,b_k^w(w)\,t_k(y)$$

or using more elaborate tensor contractions and modularity for multi-parametric or multi-fidelity problems (Weihs et al., 29 Oct 2025).

• Ensembles and Mixtures of Experts: Stacking multiple trunk networks (ensemble DeepONet) or constructing patch partitions with compactly supported local trunks (PoU-MoE), balancing global and local representation power and improving generalization in resolving multi-scale phenomena (Sharma et al., 20 May 2024).

A table summarizing key DNO architecture classes is provided below:

Architecture	Key Feature	Applications
DeepONet	Branch/trunk decomposition	Generic operator learning
FNO	Fourier convolution kernels	Periodic, translation-invariant PDEs
Ensemble DeepONet	Multiple trunk bases	Multi-scale/multi-physics problems
MNO/MONet	Multi-operator, multi-input	Parametric/multi-task learning
D2NO	Distributed, sensor-heterogeneous	Multi-physics, irregular data

5. Application Domains and Numerical Performance

DNOs have demonstrated robust performance across computational mechanics, fluids, control, and beyond:

• Porous media flow: FNO achieves ≈9% error for Darcy flow over large operator families, DeepONet ≈2% on complex geometries (Goswami et al., 2022).
• Navier–Stokes equations: DeepONet and its variants achieve 0.33–1.2% error for cavity flow and reproduce key flow patterns (e.g. vortex shedding) (Rajagopal et al., 2023).
• Inverse problems: Physics-informed DNOs discover unknown PDE terms or infer hidden system parameters with $\mathcal{O}(10^{-2})$ solution errors and $\mathcal{O}(10^{-3})$ absolute parameter errors on benchmarks including Burgers’ and reaction-diffusion equations (Kag et al., 6 Dec 2024).
• Digital twins: FNO surrogates for additive manufacturing predict melt-pool fields at 0.8% relative error, enabling millisecond-scale control loops and real-time parameter calibration (Liu et al., 13 May 2024).

A cross-section of representative accuracies is given below (all as relative test errors):

Problem	Model	Relative Error
2D Darcy flow	FNO	9.3%
	DeepONet	2.02%
Lid-driven cavity flow	POD-DeepONet	0.33%
Porous flow, no labels	PI-DeepONet (variational)	3.12%
Brittle fracture, 11 samples	PI-DeepONet	2.16%
Ocean surface forecasting	FourCastNet	5–50% of variability
AM Digital Twin (L-PBF)	FNO	0.8%

6. Implementation, Training, and Practical Guidelines

• Training data: Simulations or experimental measurements for pairs of input/output functions, chosen to span the expected function space.
• Optimization: Adam/AdamW optimizers, learning rates $10^{-3}$–$10^{-5}$, weight decay, PTR-LR schedules, and loss balancing between data and physics-residual terms.
• Regularization: Spectral normalization, weight decay, and residual/partition-of-unity (PoU) modules help ensure stability, especially for deep/trunk-heavy architectures (Sharma et al., 20 May 2024).
• Sensor selection: For heterogeneous data or complex domains, partition input spaces and adapt sensor location/coverage to the local regularity, as in D2NO (Zhang et al., 2023).
• Deployment: Once trained, DNOs enable rapid "train-once, infer-anywhere” surrogacy. Export for use in embedded control, optimization, and digital-twin environments.

7. Limitations, Challenges, and Future Directions

While DNOs provide an expressive and efficient approach for operator learning, several challenges remain:

• Curse of dimensionality: For very high-dimensional input/output spaces, network size and sample requirements can scale unfavorably; methods such as resolution-independent dictionaries (Bahmani et al., 17 Jul 2024), multi-resolution analysis, or domain decomposition are active areas.
• Operator bias/inductive priors: Selection of appropriate features, architecture (e.g., Fourier, graph, or physics-guided layers), and regularization critically impacts extrapolation and generalization.
• Uncertainty quantification: Bayesian neural operators, ensembles, and propagation of epistemic/aleatoric uncertainties through surrogates are under current investigation (Liu et al., 13 May 2024).
• Integration with optimization and control: Solver-in-the-loop, training for policy iteration in stochastic control/HJB contexts, and embedding in predictive control frameworks are recent trends (Jong et al., 23 May 2025, Lee et al., 16 Jun 2024).

DNOs are increasingly constituting a foundational computational toolset in data-driven scientific modeling, with strong mathematical guarantees, algorithmic versatility, and growing empirical success across multi-physics, multi-scale, and real-time inverse problems.