Spectral-Embedded DeepONet (SEDONet)

• The paper introduces SEDONet, which integrates spectral embeddings into DeepONet to overcome low-frequency bias in operator learning.
• SEDONet employs fixed Chebyshev and adaptive SVD-based embeddings to capture boundary layers and multiscale phenomena in PDEs.
• Performance benchmarks show SEDONet achieves 30–40% improvements in error reduction over baseline models in diverse PDE applications.

Spectral-Embedded DeepONet (SEDONet) is a data-driven operator learning framework that augments the classical Deep Operator Network (DeepONet) architecture with spectral structure, yielding improved accuracy, spectral fidelity, and robustness for nonlinear operator approximation—especially in partial differential equations (PDEs) exhibiting non-periodic, boundary-layer, or multiscale phenomena. SEDONet can refer to either networks with fixed Chebyshev spectral trunk embeddings (Abid et al., 9 Dec 2025), as well as more general strategies that extract orthonormal, hierarchical custom polynomial bases via singular value decomposition (SVD) and covariance analysis (Meuris et al., 2021). These spectral embeddings mitigate the inherent low-frequency bias of vanilla fully connected trunks and address deficiencies of periodic feature mappings in bounded or non-periodic domains.

1. Classical DeepONet Formulation and Limitations

The prototypical DeepONet approximates a nonlinear operator $\mathcal{G} : \mathcal{U} \to \mathcal{S}$, mapping input functions $u$ (e.g., PDE data) to output fields, via a dual branch–trunk factorization: $$\mathcal{G}_\theta(u)(x) = \sum_{k=1}^p b_k(u)\;\tau_k(x),$$ where the branch network $b_k(u)$ extracts input-dependent coefficients, and the trunk network $\tau_k(x)$ outputs basis functions at spatial or spatio-temporal coordinates $x$. Both networks are traditionally implemented as multilayer perceptrons (MLPs). This architecture fits a wide array of operator learning scenarios but is spectrally biased: MLP trunks acting on raw coordinate inputs preferentially represent low frequencies and smooth backgrounds, thereby under-resolving sharp gradients, boundary layers, and non-periodic features dominant in many physical systems (Abid et al., 9 Dec 2025).

2. Spectral Embedding With Chebyshev Polynomial Trunks

The Chebyshev spectral dictionary addresses this bias by embedding coordinates $x$ into a predetermined vector of Chebyshev polynomial values prior to any trainable trunk layers. For physical coordinates $x\in[x_{\min},x_{\max}]$, the transformation is

$$z = \frac{2x - (x_{\min}+x_{\max})}{x_{\max}-x_{\min}},\quad z\in[-1,1],$$

allowing the spectral feature

$$\Phi_{\mathrm{Cheb}}(x) = [T_0(z),T_1(z),\dots,T_K(z)]^\top,$$

where $T_n(z)=\cos(n\arccos(z))$ are Chebyshev polynomials of degree $n$. This embedding is strictly parameter-neutral: the transformation adds no new trainable weights.

The trunk network receives $\Phi_{\mathrm{Cheb}}(x)$ and synthesizes basis channels $t_k(x)$ via a small MLP: $$t_k(x)\approx\Psi_\theta(\Phi_{\mathrm{Cheb}}(x))_k,\quad k=1,\ldots,p,$$ and the overall network output becomes

$\hat{u}(x) = \sum_{k=1}^p b_k(u)\,t_k(x).$

This architecture enables a robust inductive bias toward non-periodic, boundary-localized structure, directly reflecting the properties of Chebyshev polynomials (Abid et al., 9 Dec 2025).

3. Construction of Orthonormal, Hierarchical Custom Bases

An alternative spectral embedding strategy is to extract custom basis functions by analyzing candidate outputs from a trained DeepONet trunk network. After training $\mathcal{G}_{\mathrm{NN}}$ on $(u_0^{(m)},u^{(m)}(t_n,x_i))$ pairs, the trunk net provides a collection of candidate spatial functions $\tau_k(x)=\gamma_k(t_0,x)$ (sampled at selected times $t_j$). The $L^2$ covariance operator over $\{\tau_k\}$ is constructed as

$\mathcal{C} = \sum_{k=1}^p \tau_k \otimes \tau_k,$

whose eigenfunctions $\{\phi_j\}$, ordered by eigenvalue $\sigma_j^2$, yield an orthonormal hierarchical basis. Practically, SVD is computed on the matrix $B=W^{1/2}A$, where $A_{ik}=\tau_k(x_i)$ and $W$ are quadrature weights: $$B = QSV^\top,\quad \phi_j(x_i) \approx (W^{-1/2} Q)_{ij}.$$ Selecting the top $r$ modes above a singular-value threshold gives a PDE-adapted orthonormal custom spectral basis, generalizing classical Fourier or polynomial systems and aligning with the underlying data distribution (Meuris et al., 2021).

4. Training Methodologies and Hyperparameter Choices

Both spectral-embedded DeepONet strategies preserve the original branch–trunk synthesis and training dynamics. Standard choices include:

• Branch network: MLP, typically depth 4, width $100$–$200$, $\tanh$ or GeLU activation.
• Trunk: MLP, depth 3–4, width $100$, $\tanh$ activation.
• Spectral embedding: Chebyshev degree $K=20$–$200$ depending on spatial resolution or SVD-derived number of modes $r$.
• Loss function: Mean squared error over all function–coordinate pairs,

$$\mathcal{L}(\theta) = \frac{1}{NQ} \sum_{i=1}^N \sum_{q=1}^Q |\hat{u}^{(i)}(x^q) - u^{(i)}(x^q)|^2,$$

• Optimizer: Adam, initial learning rate $10^{-3}$, batch sizes $32$–$256$.

No additional regularization beyond early stopping is required, and spectral embedding incurs negligible computational overhead. Chebyshev features can be rapidly evaluated (recurrence in $\mathcal{O}(K)$), and custom SVD-based bases are produced via offline linear algebra (Abid et al., 9 Dec 2025, Meuris et al., 2021).

5. Performance Benchmarks and Spectral Analysis

SEDONet delivers state-of-the-art performance in data-driven surrogate modeling for canonical PDE systems, including 2D Poisson, 1D Burgers, advection–diffusion, Allen–Cahn dynamics, and the Lorenz-96 chaotic system. Empirical evaluation shows that SEDONet achieves the lowest average relative $L^2$ errors and minimal variance compared to vanilla DeepONet and Fourier-embedded variants: | Dataset | DeepONet | FEDONet | SEDONet | |----------------------------|------------|-----------|--------------| | 2D Poisson | 1.39% | 1.10% | 0.97% | | 1D Burgers | 6.20% | 4.47% | 3.98% | | 1D Advection–Diffusion | 8.00% | 4.67% | 4.33% | | Lorenz-96 (chaotic ODE) | 23.63% | 22.03% | 20.90% | | Allen–Cahn | 10.01% | 5.70% | 5.44% |

SEDONet demonstrates 30–40% improvement over DeepONet and consistent gains over FEDONet, particularly for non-periodic, bounded domains (Abid et al., 9 Dec 2025). Spectral analyses reveal that SEDONet, leveraging Chebyshev embedding, more accurately preserves high-frequency energy and boundary-localized structure, whereas DeepONet exhibits low-frequency bias and oscillatory error near boundaries. FEDONet (with random Fourier features) excels for periodic domains but shows boundary mismatches in Dirichlet problems (Abid et al., 9 Dec 2025, Sojitra et al., 15 Sep 2025).

6. Comparison to Fourier-Embedded and Data-Driven Spectral Methods

SEDONet subsumes both fixed polynomial (Chebyshev) and adaptive custom-basis (SVD-based) spectral trunk modifications. FEDONet (Sojitra et al., 15 Sep 2025) employs random Fourier feature mappings $$\phi(\zeta) = [\sin(2\pi B \zeta), \cos(2\pi B \zeta)]$$ to inject periodic spectral content, strictly enlarging the trunk hypothesis space and enabling superior multiscale resolution for periodic or oscillatory PDEs. Ablation on Fourier feature dimension $M$ and variance $\sigma$ indicates a trade-off between expressivity and conditioning, with $M=256$, $\sigma=10$ giving robust empirical performance. SEDONet's non-periodic polynomial embedding is optimally suited for domains with boundaries and non-periodic structure (Abid et al., 9 Dec 2025).

Custom-basis SEDONet, constructed via SVD, yields hierarchical, orthonormal basis functions explicitly tuned to the PDE under study. This enables spectral Galerkin projection and temporal extrapolation well beyond the DeepONet training window, matching or exceeding the accuracy of classical Fourier bases while providing added flexibility in basis selection, boundary handling, and extrapolation capacity (Meuris et al., 2021).

7. Extensions, Limitations, and Future Directions

SEDONet architectures can be generalized via:

• Alternative spectral embeddings (wavelets, Legendre) for localized or mixed-domain problems.
• Hybrid tensor-product spectral dictionaries for multi-dimensional and spatio-temporal domains.
• Basis reuse across PDE families and operator settings.
• Trainable frequency dictionaries for adaptive spectral alignment (Sojitra et al., 15 Sep 2025).

Notable limitations include:

• Triple-product integrals for nonlinear terms may incur substantial computational expense due to the lack of fast inverse transforms.
• Extensions to higher-dimensional or complex geometries require robust partition-of-unity or interpolation architectures.
• Shock-dominated dynamics (e.g., inviscid Burgers) demand additional memory terms for stable projections and energy dissipation.

Promising directions include memory-augmented closure models, accelerated basis transforms via secondary DeepONet surrogates, and seamless applicability to static PDE operators mapping boundary or forcing profiles to solutions (Meuris et al., 2021, Abid et al., 9 Dec 2025, Sojitra et al., 15 Sep 2025).

SEDONet integrates classical spectral method principles with modern neural operator frameworks, producing robust, efficient, and accurate surrogates tailored to the spectral features of the target system.