Spectral & Spatial Neural Operator (S2NO)

• Spectral and Spatial Neural Operator (S2NO) is a framework that integrates global frequency priors with local spatial aggregation to learn mappings between function spaces.
• It achieves discretization invariance and generalizes across graphs, meshes, and irregular grids by combining spectral convolutions with localized attention mechanisms.
• S2NO demonstrates empirical success in solving PDEs, optimizing material designs, and performing graph learning while ensuring theoretical benefits like transferability and interpretability.

A Spectral and Spatial Neural Operator (S2NO) is a general neural framework for learning mappings between function spaces by simultaneously exploiting spectral and spatial representations of the underlying domain, such as a graph, grid, mesh, or Euclidean region. This scheme unifies the advantages of spectral-domain designs—access to global frequency priors, expressivity, and transferability—with the flexibility and locality of spatial, attention, or message-passing mechanisms. S2NO architectures have been rigorously formalized within graph neural networks (Balcilar et al., 2020), operator learning for PDEs on irregular domains (Sarkar et al., 2024, Sarkar et al., 13 Aug 2025), functional inverse design for morphing materials (Chen et al., 16 Jan 2026), interpretable pseudo-differential symbolic learning (Lee et al., 20 Sep 2025), and hybrid wavelet–Fourier transformers (Zhou et al., 24 Nov 2025). Typical S2NO layers integrate spectral convolutions or global filters in a geometric basis with learned spatial kernels or localized attention, producing models that are discretization-invariant, generalizable across resolutions and topologies, and capable of capturing both global and local behaviors.

1. Mathematical Formulation and Unified Operator Framework

The S2NO formalism begins by representing the computational domain (a Euclidean region, mesh, graph, or point cloud) in two complementary ways:

• Spectral Representation:

Given a Laplacian operator (continuous or discrete) $L = U \Lambda U^\top$, the eigenpairs $(\lambda_n, \phi_n)$ yield an orthonormal basis (Fourier, Chebyshev, wavelet, or Laplacian eigenfunctions). Input fields are projected onto this basis:

$u(x) = \sum_{n=1}^N \hat u_n \phi_n(x)$

Spectral convolution is defined by a learned diagonal filter $g_\theta(\lambda)$:

$$w(x) = \sum_{n=1}^N g_\theta(\lambda_n) \hat u_n \phi_n(x)$$

• Spatial Kernel/Message-Passing:

Local information is aggregated either through graph convolutions (e.g., sum over neighbors, weighted by learned gates and spatial distances) or mesh-based convolutions, producing spatially localized updates.

Unified S2NO layers fuse these modalities, producing outputs of the form:

$$h^{(l+1)} = \sigma \left( W^{(l)} h^{(l)} + \sum_{s=1}^S \Phi_s^{(l)} \star h^{(l)} \right)$$

where $W^{(l)}$ handles local mixing, and each $\Phi_s^{(l)}$ applies a global spectral filter; $\star$ denotes a convolution or spectral transform (Sarkar et al., 2024, Chen et al., 16 Jan 2026, Sarkar et al., 13 Aug 2025).

Key architectural variants include:

• Joint parameterization in $x$ and $\xi$ (position and frequency), e.g., KANO’s pseudo-differential symbol $p(x, \xi)$ (Lee et al., 20 Sep 2025).
• Gated or fused concatenation of parallel spatial and spectral branches, with channel-wise mixing (Sarkar et al., 2024, Sarkar et al., 13 Aug 2025, Chen et al., 16 Jan 2026).
• Hybrid spectral-attention schemes mixing Fourier (global) and wavelet (local) features (Zhou et al., 24 Nov 2025).

2. Spectral Analysis and Spatial Correspondence

The equivalence between spectral and spatial graph convolutions is established formally in (Balcilar et al., 2020):

• Any spectral filter $g_\theta(\Lambda)$ in the Laplacian basis corresponds to a spatial kernel $C = U \mathrm{diag}(g_\theta(\Lambda)) U^\top$.
• Frequency profiles $\gamma_s(\lambda)$ are transformed into spatial supports $C^{(s)}$; this allows for arbitrary bandpass, low-pass, or high-pass behavior, with spatial localization determined by the smoothness of $\gamma_s(\lambda)$.

Depthwise-separable parameterizations further reduce complexity:

$$H^{(l+1)} = \sigma \left( \left( \sum_{s=1}^S w^{(s,l)} \odot [C^{(s)} H^{(l)}] \right) W^{(l)} \right)$$

where $w^{(s,l)}$ are scalar channel gates and $W^{(l)}$ is a mixing matrix (Balcilar et al., 2020).

Band-specific filters, spectral gating, and fusion strategies are systematically shown to improve expressivity and handle over-smoothing or over-squashing effects in deep architectures (Sarkar et al., 2024, Sarkar et al., 13 Aug 2025).

3. Architecture Design and Scalability

S2NO architectures consist of repeated blocks with parallel spectral and spatial branches, channel fusion, and pointwise nonlinearities. Representative pipeline components (with notation from (Chen et al., 16 Jan 2026, Sarkar et al., 2024, Sarkar et al., 13 Aug 2025)) include:

1. Input Lifting: Linear or MLP transformation $P$ from raw input features to feature channels.
2. Spectral Branch: Truncation to $r \ll N$ leading eigenfunctions, followed by neural filtering:

$$\mathbf{S}_m \left( \mathbf{K} \times_1 \left( \mathbf{S}_m^\top v_j + w(v_j) \right) \right)$$

where $\mathbf{S}_m$ stacks eigenvectors and $\mathbf{K}$ is a learned tensor (Sarkar et al., 13 Aug 2025, Sarkar et al., 2024).

1. Spatial Branch: Message-passing, often gated by attention or MLPs on edge features and positional embeddings:

$$v_{j+1,u}^{\text{spatial}} = \sum_{v \in \mathcal{N}(u)} \gamma_{uv} W^{(j)} v_{j,v}$$

(Sarkar et al., 13 Aug 2025, Sarkar et al., 2024).

1. Fusion/Concatenation: Channel-wise concatenation and linear mixing $W_c^{(j)}$, or a sigmoid-gated convex combination (Zhou et al., 24 Nov 2025).
2. Residual and Feedforward: LayerNorm, GeLU activation, and small MLPs for refinement (Chen et al., 16 Jan 2026).

Spectral eigen-decomposition is often approximated (e.g., Chebyshev expansions) for scalability ($O(N^3)$ for full EVD, $O(mE)$ for truncated GFT) (Balcilar et al., 2020, Sarkar et al., 2024).

Discretization or mesh invariance is achieved by re-solving the Laplacian eigenproblem on new grids and re-projecting input features, enabling zero-shot super-resolution and multi-geometry generalization (Chen et al., 16 Jan 2026, Zhou et al., 24 Nov 2025).

4. Theoretical Properties: Expressivity, Transferability, and Interpretability

Key theoretical guarantees for S2NO models include:

• Universal Representability: Any graph convolutional kernel designed in spectral or spatial domain can be realized in the other domain. Chebyshev, CayleyNet, and B-spline filters admit spatial equivalents (Balcilar et al., 2020).
• Transferability: Because spectral profiles $\gamma_s(\lambda)$ depend only on spectra, learned filters transfer across graphs or meshes of differing size and topology (Balcilar et al., 2020, Sarkar et al., 2024, Lee et al., 20 Sep 2025).
• Mesh Invariance: S2NO models trained on coarse grids generalize to fine grids without retraining, as shown in super-resolution experiments for morphing materials (Chen et al., 16 Jan 2026) and operator learning benchmarks (Zhou et al., 24 Nov 2025).
• Interpretability: When spectral/spatial symbols and nonlinearities are implemented as Kolmogorov–Arnold networks (KANs), symbolic expressions can be directly recovered for operator coefficients, facilitating closed-form extraction of PDE terms (e.g., $x^2$, $-\xi^2$, $f^3$) (Lee et al., 20 Sep 2025).
• Complexity: Depthwise-separable S2NOs reduce parameter count from $S f_l f_{l+1}$ to $S f_l + f_l f_{l+1}$ per layer (Balcilar et al., 2020).

5. Applications and Empirical Performance

S2NO frameworks have robustly demonstrated efficacy in diverse scientific and engineering contexts:

• PDE Solution Operators: Solving stationary and time-dependent equations (Poisson, Darcy, elasticity, Burgers, Allen–Cahn, Navier–Stokes) on both regular and irregular domains (Sarkar et al., 2024, Sarkar et al., 13 Aug 2025, Zhou et al., 24 Nov 2025).
• Functional Inverse Design: Material-to-shape mapping, shape-morphing programming, and optimization via evolutionary algorithms on porous, thin-walled, and multi-geometry domains (Chen et al., 16 Jan 2026).
• Graph Learning: Node classification (Cora, Citeseer, PubMed), graph classification (ENZYMES, PROTEINS, PPI), community detection, and molecular property prediction (Balcilar et al., 2020).
• Quantum Hamiltonian Learning: Symbolic reconstruction of position-dependent potentials and differential operators with accuracy to four decimal places (Lee et al., 20 Sep 2025).

Benchmark results (relative MSE, test accuracy, state infidelity) consistently show S2NO models outperform state-of-the-art baselines in both accuracy and computational efficiency, generalizing to unseen resolutions and geometries (Chen et al., 16 Jan 2026, Sarkar et al., 2024, Zhou et al., 24 Nov 2025, Balcilar et al., 2020, Lee et al., 20 Sep 2025, Sarkar et al., 13 Aug 2025).

6. Limitations, Implementation Strategies, and Future Directions

Principle constraints and technical considerations for S2NO deployments are as follows:

• Computational Cost: Full eigen-decomposition scales badly ($O(n^3)$); truncated or polynomial approximations are preferred for large domains (Balcilar et al., 2020, Sarkar et al., 2024).
• Manual Spectral Profile Design: The selection of $\gamma_s(\lambda)$ typically requires domain expertise; automated mechanisms for profile learning remain under development (Balcilar et al., 2020).
• Directed Graphs/Continuous Edge Features: Current formulations do not natively handle directed graphs or variable edge weights; generalization requires bespoke spectral bases or composite kernels (Balcilar et al., 2020).
• Sparse Hardware Utilization: Dense spatial kernels limit potential for hardware acceleration; compression and localization strategies are essential for scalability (Balcilar et al., 2020, Sarkar et al., 2024, Sarkar et al., 13 Aug 2025).
• Physics-Informed Training: For PDE learning, physics-aware losses (residual- and boundary-constrained), hybrid time-marching schemes, and stochastic projection of derivatives are critical for robust generalization (Sarkar et al., 13 Aug 2025).

Ongoing research directions include fully adaptive graph learning, multi-scale spectral grids, improved symbolic extraction, and hybridization of spectral bases (wavelets, Chebyshev) in the S2NO blueprint (Lee et al., 20 Sep 2025, Zhou et al., 24 Nov 2025).

7. Comparative Summary of Principal S2NO Variants

Model/Framework	Domain	Fusion Mechanism	Key Benchmarks
DSGCN S2NO (Balcilar et al., 2020)	Graph	Depthwise-separable spectral	Cora, PPI, ENZYMES
Sp²GNO (Sarkar et al., 2024)	Graph/PDE	Parallel spectral+spatial, fuse	Elliptic, Elasticity, Airfoil
πG-Sp²GNO (Sarkar et al., 13 Aug 2025)	Graph/PDE	Spectral+spatial, geometry-aware	Poisson, Darcy, Plate, Burgers
S2NO (Chen et al., 16 Jan 2026)	Mesh/morphing	Laplacian spectral + gated spatial	Shape-morphing, super-resolution
KANO (Lee et al., 20 Sep 2025)	Fourier/PDE	Symbolic $(x,\xi)$ KAN fusion	Quantum Hamiltonians, symbolic PDE
SAOT (Zhou et al., 24 Nov 2025)	Grid/PDE	Gated fusion FA+WA	Darcy, Elasticity, Navier–Stokes

These implementations demonstrate that S2NOs afford a unified, mesh-invariant operator learning framework that effectively integrates spectral prior knowledge with local adaptability, delivering optimal performance for multiscale scientific and geometric learning problems.