Self-Supervised Neural Operator (SNO)

• SNO is a self-supervised framework that learns mappings between function spaces by exploiting internal physics-based and variational constraints.
• It integrates physics-informed samplers, encoder-transformers, and NTK-based adaptive weighting to generate training data on the fly and boost prediction accuracy.
• SNO is applied across diverse domains including PDE solving, optimal control, Wasserstein gradient flows, time-series learning, and functional clustering.

Self-Supervised Neural Operator (SNO) denotes a family of neural-operator methods in which the supervisory signal is derived from structure internal to the problem rather than from paired input-output labels. In the PDE setting, this usually means learning a solution operator from governing equations, boundary or initial conditions, and collocation-based residuals; in other settings it can mean learning from objective functionals, variational principles, flow-matching dynamics, or clustering consistency. The label is used most directly in “Self-supervised neural operator for solving partial differential equations” (You et al., 31 Aug 2025), where SNO is proposed as a neural-operator framework that “generates accurate and diverse training data on the fly without numerical solvers,” and more broadly in adjacent work on physics-informed DeepONets, amortized optimal-control operators, JKO operators for Wasserstein gradient flow, time-series representation learning, and functional clustering (Wang et al., 2021, Wang et al., 2021, Xu et al., 31 Dec 2025, Feng et al., 9 Jan 2026, Nguyen et al., 12 Feb 2026, Li et al., 26 Feb 2026).

1. Conceptual scope and operator-learning setting

Neural operators learn maps between function spaces rather than between fixed-dimensional vectors. A standard formulation writes an operator

$G:\mathcal U\to\mathcal V,$

mapping an input function $u$ to an output function $s=G(u)$. In PDE applications, the governing system is typically expressed as

$\mathcal N(u,s)=0,\qquad \mathcal B(u,s)=0,$

with $\mathcal N$ the PDE operator and $\mathcal B$ the boundary or initial-condition operator (Wang et al., 2021).

Within this setting, self-supervision means that training need not rely on paired samples $\{(u^{(i)},s^{(i)})\}_{i=1}^N$. Instead, the model can be optimized from residuals computed through automatic differentiation, objective functionals evaluated along simulated trajectories, or other internally generated consistency conditions. The physics-informed DeepONet literature made this point explicit by showing that solution operators of parametric PDEs can be learned “in an entirely self-supervised manner” and “without any paired input-output observations” when the governing equations and boundary conditions are known (Wang et al., 2021). A related formulation for PDE-constrained optimization similarly states that “the formulation of the physics-informed DeepONets does not require any paired input-output observations for training the model, except for a set of given boundary conditions” (Wang et al., 2021).

A modern PDE-specific SNO instantiation, as described in the abstract of (You et al., 31 Aug 2025), consists of three parts: “a physics-informed sampler (PI-sampler) based on Bayesian PINNs for efficient data generation, a function encoder (FE) for compact input-output representations, and an encoder-only Transformer for operator learning, mapping boundary/initial conditions, source terms, and geometries to PDE solutions.” This places SNO within the broader neural-operator program while shifting the bottleneck from external high-fidelity solver data to internally generated training signals.

2. PDE-oriented SNOs and physics-informed self-supervision

The most established SNO formulation is physics-informed operator learning for PDEs. In DeepONet form, the operator surrogate is commonly written as

$G_\theta(u)(y)=\sum_{k=1}^q b_k(u)\,t_k(y),$

where the branch network encodes the input function and the trunk network encodes the query coordinate (Wang et al., 2021). In the PDE-constrained optimization variant, the same branch-trunk structure is used with

$$\mathcal{G}_{\theta}(u)(y) = \sum_{k=1}^{q} b_k(u(x_1),u(x_2),\ldots,u(x_m))\, t_k(y),$$

providing a continuous representation of the output field that can be differentiated with respect to input coordinates (Wang et al., 2021).

Self-supervision arises by enforcing differential constraints directly. One representative loss decomposition is

$$\mathcal{L}(\theta)=\mathcal{L}_{BC}(\theta)+\mathcal{L}_{PDE}(\theta),$$

or, more generally,

$u$0

where the physics term enforces PDE and boundary-condition residuals at collocation points (Wang et al., 2021, Wang et al., 2021). In the fully self-supervised regime, the operator-data term can be absent, so the entire training signal is furnished by the governing equations.

This framework was extended in (Wang et al., 2021) through an NTK analysis of DeepONet training dynamics. That work identifies a “bias that favors the approximation of functions with larger magnitudes” and proposes adaptive reweighting based on the diagonal of the NTK:

$u$1

with $u$2 corresponding to “NTK weights” and $u$3 to “moderate NTK weights.” It also introduces a modified DeepONet architecture that mixes branch and trunk information at every hidden layer, and reports predictive-accuracy improvements “by a factor of 10-50x” in the absence of paired observations (Wang et al., 2021).

The 2025 SNO preprint (You et al., 31 Aug 2025) adopts a different but related strategy. Its abstract states that SNO “generates accurate and diverse training data on the fly without numerical solvers,” and validates the approach on “1D steady/unsteady nonlinear reaction-diffusion equations, a 2D nonlinear PDE with varying geometries, and vortex-induced vibration of a flexible cylinder in fluid dynamics.” It further reports that SNO “achieves high accuracy in all cases,” and that “lightweight finetuning ($u$4 trainable variables) further improves predictions with only a few hundred steps” (You et al., 31 Aug 2025). A plausible implication is that this work shifts self-supervised operator learning from direct residual minimization alone toward hybrid schemes that first synthesize training functions and then learn an operator over those functions.

3. Self-supervision mechanisms beyond PDE residuals

SNO has expanded beyond PDE residual enforcement into several distinct self-supervisory mechanisms. In self-supervised amortized optimal control, the target is not a PDE state but an optimal-control solution operator

$u$5

Training minimizes the population amortized risk

$u$6

with the trajectory generated by applying the predicted control through the dynamics. The supervision comes from the original objective functional and dynamics, “not ground-truth optimal controls” (Xu et al., 31 Dec 2025).

In Wasserstein gradient flows, the self-supervised signal is variational. The JKO operator is defined by

$u$7

and the learned operator $u$8 is trained from the one-step loss

$u$9

The “Learn-to-Evolve” algorithm alternates between trajectory generation and operator updates, so that the model “jointly learns the JKO operator and its induced trajectories” and the generated trajectories act as “a natural form of data augmentation” (Feng et al., 9 Jan 2026).

In time-series SSL, the Flow-Guided Neural Operator (FGNO) combines operator learning with flow matching. Its time-conditioned network

$s=G(u)$0

is trained on noisy interpolations

$s=G(u)$1

with objective

$s=G(u)$2

Here, self-supervision comes from denoising and vector-field prediction rather than labels, and the corruption level $s=G(u)$3 is treated as “a new degree of freedom for representation learning” (Nguyen et al., 12 Feb 2026).

In functional clustering, SNO denotes an unsupervised operator from an RKHS $s=G(u)$4 to $s=G(u)$5, with cluster regions recovered by thresholding

$s=G(u)$6

The practical model uses a frozen encoder and a trainable head, while the self-supervised loss combines consistency, confidence, and entropy terms:

$s=G(u)$7

This shifts SNO from regression to “decision regions” in function space (Li et al., 26 Feb 2026).

4. Representative architectures and application domains

The following works illustrate how the SNO label, or closely related self-supervised neural-operator methodology, has been instantiated across domains.

Work	Domain	Defining mechanism
(You et al., 31 Aug 2025)	PDE solving	PI-sampler, FE, encoder-only Transformer
(Wang et al., 2021)	PDE solution operators	Physics-informed DeepONet with NTK-guided weighting
(Wang et al., 2021)	PDE-constrained optimization	Physics-informed DeepONet as differentiable surrogate
(Xu et al., 31 Dec 2025)	Optimal control	Amortized risk over task distributions
(Feng et al., 9 Jan 2026)	Wasserstein gradient flow	Learn-to-Evolve JKO operator
(Nguyen et al., 12 Feb 2026)	Time-series SSL	Flow matching on STFT spectrograms
(Li et al., 26 Feb 2026)	Functional clustering	Sampling-based neural operator to $s=G(u)$8

In PDE forward solving, (You et al., 31 Aug 2025) targets boundary or initial conditions, source terms, and geometries as operator inputs, with validation on nonlinear reaction-diffusion equations, a 2D nonlinear PDE with varying geometries, and vortex-induced vibration of a flexible cylinder. In PDE-constrained optimization, (Wang et al., 2021) uses a trained physics-informed DeepONet as a fast differentiable surrogate for control and design variables, demonstrating applications in time-dependent optimal control of heat transfer and drag minimization of obstacles in Stokes flow.

In optimal control, (Xu et al., 31 Dec 2025) extends the SNO idea to open-loop solution operators and further embeds the learned operator into Model Predictive Control for dynamic or partially observed environments. In Wasserstein gradient flow, (Feng et al., 9 Jan 2026) learns an operator that maps an input density directly to the minimizer of the corresponding JKO subproblem and then applies it iteratively to generate the full evolution. In self-supervised time-series learning, (Nguyen et al., 12 Feb 2026) uses STFT to unify different time resolutions and extracts representations from different network layers and flow times. In clustering, (Li et al., 26 Feb 2026) applies SNO to unlabeled families of ODE trajectories and learns soft cluster assignments in function space rather than pointwise predictions.

A plausible implication is that SNO is better viewed as a methodological pattern than as a single architecture. Across these works, the common structure is an operator-learning backbone plus a self-generated training signal, but the signal itself may be physical, variational, dynamical, generative, or set-valued.

5. Theoretical results and empirical behavior

The theory of self-supervised operator learning has developed along several axes. For physics-informed DeepONets, NTK analysis yields a training-dynamics explanation for the magnitude bias in self-supervised PDE operator learning and motivates adaptive weighting to balance residual convergence rates (Wang et al., 2021). In amortized optimal control, (Xu et al., 31 Dec 2025) proves that if $s=G(u)$9 solves the population problem, then for $\mathcal N(u,s)=0,\qquad \mathcal B(u,s)=0,$0-almost every $\mathcal N(u,s)=0,\qquad \mathcal B(u,s)=0,$1,

$\mathcal N(u,s)=0,\qquad \mathcal B(u,s)=0,$2

and derives scaling laws showing that generalization depends on intrinsic dimension $\mathcal N(u,s)=0,\qquad \mathcal B(u,s)=0,$3 and regularity $\mathcal N(u,s)=0,\qquad \mathcal B(u,s)=0,$4, with asymptotic behavior governed by the exponent

$\mathcal N(u,s)=0,\qquad \mathcal B(u,s)=0,$5

The paper emphasizes a “quantifiable price to pay”: neural operators can exploit hidden low-dimensional structure, but performance degrades as intrinsic complexity increases (Xu et al., 31 Dec 2025).

For Wasserstein gradient flows, (Feng et al., 9 Jan 2026) provides convergence results under Lipschitz assumptions on the true JKO operator and the learned operator. If the training error

$\mathcal N(u,s)=0,\qquad \mathcal B(u,s)=0,$6

tends to zero, then the generated datasets converge to the true JKO datasets, and the learned trajectories converge in Wasserstein distance. In functional clustering, (Li et al., 26 Feb 2026) proves a “universal clustering theorem” stating that any finite family of closed cluster regions in an RKHS can be approximated by sampling-based neural operators in the upper Kuratowski sense.

Empirically, the reported gains are domain-specific rather than uniform across all SNO variants. The PDE-oriented SNO abstract reports “high accuracy in all cases” and improved predictions from lightweight finetuning with $\mathcal N(u,s)=0,\qquad \mathcal B(u,s)=0,$7 trainable variables (You et al., 31 Aug 2025). Physics-informed DeepONets improve predictive accuracy “by a factor of 10-50x” in the absence of paired observations (Wang et al., 2021). In PDE-constrained optimization, (Wang et al., 2021) reports a test relative $\mathcal N(u,s)=0,\qquad \mathcal B(u,s)=0,$8 error of 0.09% for a 1D Poisson surrogate, a learned control error of 0.37%, about 25 seconds total optimization time and about 5× faster than a conventional adjoint method using FEniCS for a 2D heat-control problem, and Stokes-flow test errors of 0.97% ± 0.24% for velocity $\mathcal N(u,s)=0,\qquad \mathcal B(u,s)=0,$9, 4.79% ± 0.53% for velocity $\mathcal N$0, and 1.53% ± 0.67% for pressure $\mathcal N$1.

Outside PDEs, FGNO reports “up to 35% AUROC gains in neural signal decoding (BrainTreeBank), 16% RMSE reductions in skin temperature prediction (DREAMT), and over 20% improvement in accuracy and macro-F1 on SleepEDF under low-data regimes” (Nguyen et al., 12 Feb 2026). The clustering SNO in (Li et al., 26 Feb 2026) reports, on ODE-6, ACC $\mathcal N$2, ARI $\mathcal N$3, NMI $\mathcal N$4 for SNO and ACC $\mathcal N$5, ARI $\mathcal N$6, NMI $\mathcal N$7 for SNO + spectrogram, with weaker but still leading results on the harder ODE-4 benchmark.

6. Terminology, misconceptions, and related distinctions

A recurring source of confusion is that “SNO” is not globally unambiguous. In (Li et al., 26 Feb 2026), SNO denotes a self-supervised neural operator for functional clustering. In (You et al., 31 Aug 2025), it denotes a self-supervised neural operator for solving PDEs. By contrast, (Santos et al., 28 Apr 2026) uses SNO to mean “Shearlet Neural Operator,” a supervised architecture that replaces Fourier spectral mixing with a shearlet-based representation and “does not discuss self-supervised learning, unsupervised pretraining, masked reconstruction, contrastive objectives, pseudo-labeling, [or] representation learning without PDE labels.” Any technical discussion of SNO therefore requires explicit disambiguation (Santos et al., 28 Apr 2026).

Another common misconception is that self-supervision in neural operators is synonymous with contrastive or masked-reconstruction objectives. The literature here is broader. In PDE operator learning, self-supervision often means training from PDE residuals and boundary or initial conditions alone (Wang et al., 2021, Wang et al., 2021). In optimal control, it means optimizing predicted controls directly against the original cost functional and dynamics, “not labeled optimal trajectories” (Xu et al., 31 Dec 2025). In Wasserstein gradient flow, it means minimizing the JKO variational objective without numerical JKO trajectories (Feng et al., 9 Jan 2026). In time-series representation learning, it can mean flow matching with controllable corruption levels and clean-input probing at inference (Nguyen et al., 12 Feb 2026).

The current literature also shows that SNO is not uniformly a forward-solver technology. Some SNOs approximate PDE solution operators, some amortize optimal controls, some learn transport-based variational updates, and some discover cluster regions in infinite-dimensional spaces. This suggests that the most stable definition of SNO is not architectural but procedural: a neural operator trained from internally generated supervisory structure.

Finally, the literature is explicit about limitations. The scaling-law analysis in (Xu et al., 31 Dec 2025) argues that neural operators can avoid the ambient curse of dimensionality but “remain fundamentally constrained by the intrinsic dimensional complexity in more challenging settings.” FGNO’s benefits are tied to the availability of a useful time-frequency representation and a flow-time feature hierarchy (Nguyen et al., 12 Feb 2026). The JKO framework in (Feng et al., 9 Jan 2026) depends on the stability of proximal gradient-flow structure. Physics-informed DeepONets incur overhead from NTK weighting and architectural modifications, with NTK weighting reported as roughly 3–5× slower than no weighting and the modified architecture about 2× slower than a conventional DeepONet (Wang et al., 2021). These caveats place SNO within a technically rich but still heterogeneous research area, rather than a single settled method.