Adv-NO: Adversarially Trained Neural Operator

• Adv-NO is a neural operator enhanced with adversarial training that integrates explicit adversarial losses with regression objectives to capture fine-scale features.
• It leverages methodologies such as GAN-inspired discrimination, operator norm regularization, and adversarial autoencoding to improve robustness across challenging regimes.
• Empirical studies show significant error reductions and enhanced out-of-distribution performance in applications like turbulent flow super-resolution and dynamic system forecasting.

An adversarially trained neural operator (adv-NO) is a neural operator framework enhanced with adversarial training techniques—primarily borrowed from generative adversarial networks and adversarial robustness literature—designed to improve accuracy, generalization, and robustness, especially in regimes with complex high-frequency structures or under adversarial perturbations. Adv-NOs combine the data-fitting strengths of operator learning with both explicit and implicit adversarial objectives, resulting in models that can better recover fine-scale features, generalize to out-of-distribution regimes, and resist attack or noise, including in hardware-accelerated scenarios.

1. Core Methodological Principles

The essential principle underlying adv-NO is the integration of adversarial losses (or adversarially inspired regularizations) with the operator learning framework. In its canonical form for generative learning, a neural operator (e.g., DeepONet, UNet, or a Koopman autoencoder) acts as a generator $\mathcal{G}_\theta$ and is paired with a discriminator network $D_\phi$ that judges the quality or realism of outputs in either sample or latent spaces.

The training objective typically augments the regression loss (e.g., $L_2$ or $L_1$ error) with adversarial loss terms. In the context of turbulent flow super-resolution, for example, the adv-NO is trained using a composite loss: $$\mathcal{L}_{\text{advNO}} = \| u_{\text{tar}} - u_f \|_1 + \sum_{l=1}^L w_l \| \phi_l(u_{\text{tar}}) - \phi_l(u_f) \|_1 + \beta L_{G}$$ where $u_f = \mathcal{G}_\theta(u_{\text{in}})$, $u_{\text{tar}}$ is the high-resolution target, $\phi_l$ are feature maps for perceptual loss, and $L_G$ is an adversarial loss computed using a relativistic average discriminator (Oommen et al., 10 Sep 2025).

Alternatively, adversarial regularization can be imposed at the latent level via adversarial autoencoder frameworks, where a discriminator enforces a continuous, structured latent space, as outlined for DeepONets and Koopman autoencoders. Here, encoder outputs are pushed toward predefined latent distributions, with the binary cross-entropy loss guiding the adversarial game.

In settings emphasizing adversarial robustness—rather than generative fidelity—adversarial training is theoretically linked to data-dependent operator norm regularization. Training under projected gradient ascent with $\ell_p$-constrained perturbations and $\ell_q$-norm adversarial losses equates to regularizing the $(p, q)$-operator norm of the model’s Jacobian across the relevant data manifold (Roth et al., 2019).

2. Adversarial Training in Operator Learning Architectures

Specific operator learning architectures—such as DeepONets, Koopman autoencoders, and time-conditioned UNets—have been advanced with adversarial components:

• DeepONets and Koopman Autoencoders: Adversarial additions are applied to their encoder architectures. The encoder produces a latent representation, and a discriminator forces this representation to match a smooth reference distribution (often Gaussian). Noise is further injected into latent codes before adversarial evaluation, with a typical variance scaling factor (e.g., 0.025 of empirical latent standard deviation) (Enyeart et al., 10 Dec 2024).
• Time-Conditioned UNet Operator (for Fluid Mechanics): Used in super-resolution and forecasting settings, the generator is trained with both perceptual and adversarial losses, the latter using a relativistic discriminator comparing generated and ground-truth high-resolution outputs. This design enforces recovery of high-wavenumber energy, suppressing the spectral bias induced by regression-only objectives (Oommen et al., 10 Sep 2025).
• Conditional Generative Operator Frameworks: In the sparse flow reconstruction regime, adv-NO is coupled with mask-aware conditioning and a downstream conditional generative model trained to match both the mean and higher-order (statistical, phase) properties of full-field turbulent flows.

3. Theoretical Underpinnings: Robustness and Spectral Regularization

Adv-NO's adversarial elements are supported by two main theoretical strands:

• Operator Norm Regularization: Adversarial training is, in certain regimes, equivalent to minimization of the data-dependent operator norm of the model's Jacobian. Under a local linearization, maximizing an adversarial loss with $\ell_p$ constraints is, in the power method limit ($\alpha \to \infty$), computing the dominant operator norm direction. Thus, adversarial training can be interpreted as controlling the network’s sensitivity to input perturbations; robust models have dampened singular value spectra and increased local linearity (Roth et al., 2019).
• Generative Modeling of Noise: Implicit generative modeling of random noise during training (Noise-based Prior Learning, NoL) introduces learnable multiplicative noise to inputs, with joint optimization of network parameters and noise templates acting as an implicit prior over the data manifold. This approach expands the region of effective training and aligns noise updates with adversarial directions, thereby providing intrinsic robustness and lower transferability for black-box attacks (Panda et al., 2018).

4. Applications and Empirical Outcomes

Adv-NO architectures have demonstrated utility in several domains:

Application Domain	Reported Adv-NO Benefit	Source
Schlieren jet super-resolution	15× energy-spectrum error reduction, sharp gradients	(Oommen et al., 10 Sep 2025)
3D HIT turbulence forecasting	Accurate forecasts with only 160 timesteps, 114× speedup	(Oommen et al., 10 Sep 2025)
Sparse 3D velocity/pressure recon	Phase-correct, statistics-correct reconstructions	(Oommen et al., 10 Sep 2025)
Dynamical systems (e.g., pendulum)	20–26% error reduction with adversarial autoencoding	(Enyeart et al., 10 Dec 2024)
KdV/Burgers’ PDEs (DeepONet)	4–9% error reduction under data scarcity	(Enyeart et al., 10 Dec 2024)

• In turbulent flow super-resolution, the adv-NO outperforms standard neural operators by dramatically reducing high-wavenumber error, matching the true physical spectra and maintaining neural operator-level inference costs.
• For forecasting, recursive application of the adv-NO provides stable, physically consistent turbulent evolution over long horizons with significant computational gains.
• In settings where only sparse measurements are available, adv-NO architectures paired with conditional generative models reconstruct full volumetric fields with correct phase information and statistics, including invariants like the $Q$–$R$ joint PDF (Oommen et al., 10 Sep 2025).

In scenarios with limited training data, adversarial regularization on the latent operator space (e.g., via an adversarial autoencoder loss) enhances generalization (Enyeart et al., 10 Dec 2024).

5. Hardware Effects and Adversarial Robustness

Adversarially trained neural operators deployed on non-volatile memory (NVM) analog crossbars exhibit notable dual behavior (Tao et al., 2021):

• Noise Stability: Adv-NO deployments on analog accelerators can experience up to 2× accuracy degradation on unperturbed (“clean”) inputs due to lower signal-to-noise ratio (SNR) and higher sensitivity to hardware-induced noise compared to vanilla training. This is attributed to “hardened” internal representations from adversarial training (quantified via SNR, layer cushion $\mu_i$, and noise sensitivity).
• Hardware-augmented Robustness: When attacked with non-adaptive black-box or strong white-box perturbations (e.g., Square Black Box or high-$\varepsilon$ PGD), the same crossbar non-idealities provide up to 20–30% improved robust accuracy (black-box) and 5–10% in white-box cases, when the attack strength exceeds adversarial training regularization.
• Calibration Requirement: Achieving optimal performance and robustness requires co-design. The adversarial training parameter ($\varepsilon_{\text{train}}$) and analog hardware design parameters (crossbar size, non-ideality factor) must be calibrated, as higher adversarial training magnitude reduces noise stability but can, when matched well to hardware characteristics, provide enhanced out-of-distribution robustness.

6. Analysis of Internal Representations and Out-of-Distribution Generalization

Principal Component Analysis (PCA) and complexity theoretic measures have been used to analyze the effects of adversarial training in neural operators:

• PCA Variance: Models incorporating adversarial elements (either explicit, as in GAN, or implicit, as in NoL) have higher variance along top-ranked principal components of internal activations. This is correlated with broader data manifold coverage, lower divergence between clean and adversarial feature representations, and improved robustness (Panda et al., 2018).
• Out-of-distribution Guarantees: For high-frequency PDE problems (e.g., Helmholtz equation), enhancements in neural operator design—such as the introduction of stochastic depth—improve out-of-distribution risk bounds, with analysis linked to Rademacher complexity. Stochastic perturbations in operator depth act as a regularizer for generalization in operator learning (Benitez et al., 2023).
• Latent Distribution Regularity: For autoencoding architectures, discriminator-driven adversarial regularization ensures the latent space is continuous and fully utilized, promoting better out-of-distribution and data-efficient generalization, particularly in small sample regimes (Enyeart et al., 10 Dec 2024).

7. Limitations and Trade-offs

• Trade-offs: Adversarial regularization improves robustness and high-frequency reconstruction but can reduce noise stability (critical when mapping to analog accelerators) and sometimes slightly hamper performance with abundant training data (Tao et al., 2021, Enyeart et al., 10 Dec 2024).
• Scope of Robustness: Approaches such as NoL deliver particularly improved defense against black-box attack transferability but may be insufficient alone against strong white-box attacks—necessitating combinations with explicit adversarial example augmentation or adversarial autoencoding (Panda et al., 2018).
• Resource and Implementation Considerations: Achieving the benefits of adv-NO requires careful design of training losses (balancing regression, adversarial, and perceptual terms), calibration to the target hardware, and monitoring of internal spectral and latent structure throughout training.

Summary

Adversarially trained neural operators provide a mechanism to overcome the limitations of standard operator learning with respect to spectral bias, generalization, and adversarial robustness. By combining explicit adversarial losses—at the output, feature, or latent space level—with implicit regularization of internal sensitivities or data manifold coverage, adv-NOs perform superior reconstruction, forecasting, and control in physical and experimental domains, particularly for turbulent and underdetermined systems. These hybrid strategies are especially notable in real-time or resource-constrained scenarios, as well as when robustness to attack or hardware noise is paramount (Panda et al., 2018, Roth et al., 2019, Tao et al., 2021, Benitez et al., 2023, Enyeart et al., 10 Dec 2024, Oommen et al., 10 Sep 2025).