Frequency-Gated Loss: Concepts & Applications

• Frequency-Gated Loss is a class of loss functions that employ explicit frequency-domain gating via Fourier transforms to selectively penalize prediction errors.
• It utilizes spectral weight masks, radially-binned error aggregation, and frequency-dependent gain–loss profiles to enforce targeted regularization across low, mid, and high frequencies.
• Applications span adversarial defense in computer vision, neural operator training in scientific machine learning, and optical soliton stabilization in photonics.

Frequency-Gated Loss (FGL) denotes a class of loss functions that introduce explicit frequency-domain weighting or windowing to selectively penalize prediction errors in specific frequency bands. Unlike standard spatial-domain losses, which treat all frequencies isotropically, FGL mechanisms "gate" the error signal—often privileging certain frequency components (e.g., low, mid, or high bands), according to application-specific priors such as spectral bias of neural networks, data-specific frequency distribution, or physical relevance in the target domain. This framework has emerged as a powerful regularization and robustness tool across adversarial defense in computer vision, scientific machine learning (notably neural operator training), and photonics, where spectral selectivity directly impacts stability and propagation dynamics.

1. Mathematical Formulations of Frequency-Gated Loss

FGL variants implement frequency gating through Fourier decomposition, band selection/biasing, and weighted aggregation of spectral-domain errors. In adversarial video purification, FGL is defined using the real-valued 2D discrete Fourier transform (RDFT):

$$\mathcal{L}_{\rm FGL}(\theta) = \mathbb{E}_{t,\mathbf{x}_0,\mathbf{x}_1} \Bigl\|\,\mathbf{W}\,\odot\, \bigl|\mathcal{F}\bigl(v_\theta(\mathbf{x}_t,t)\bigr)\bigr| - \bigl|\mathcal{F}(\mathbf{u}^\star)\bigr| \Bigr\|_2^2 ,$$

where $\mathbf{W}$ is a frequency-domain weight mask decaying exponentially from the DC component, $\mathcal{F}$ is RDFT, $\odot$ denotes elementwise multiplication, and $v_\theta(\mathbf{x}_t, t)$ is the predicted velocity field. The mask $\mathbf{W}\in \mathbb{R}^{H\times W'}$ is constructed as $w_{i,j} = \exp(-\tau d_{i,j}) + 0.1$ with $d_{i,j}$ the normalized distance from $(0,0)$ and $\tau$ controlling decay (Tang et al., 5 Jan 2026).

In spectral neural operator training, the "frequency-gated loss" is constructed on radially-binned spectral errors: $$\mathcal{C}_{\rm freq} = \mathbf{w} \cdot \mathbf{E}_{\rm band} = w_{\rm low} E_{\rm low} + w_{\rm mid} E_{\rm mid} + w_{\rm high} E_{\rm high} ,$$ where $\mathbf{E}_{\rm band}$ aggregates spectral error energy per bin, and weights $\mathbf{w}$ enforce the target spectral bias (e.g., $(0,1,1)$ to penalize mid/high but not low bands) (Kalimuthu et al., 5 Apr 2025).

In optical soliton stabilization, the frequency-gated mechanism is implemented through a frequency-dependent linear gain–loss profile: $$g(\omega, z) = -g_L + (g_0 + g_L) [\tanh(p[\omega + \beta(z) + W/2]) - \tanh(p[\omega + \beta(z) - W/2])]$$ enforcing low loss (even slight gain) within a band $|\omega+\beta(z)|<W/2$ tracking the soliton's instantaneous carrier, and strong loss elsewhere (Peleg et al., 2024).

2. Frequency Gating Strategies and Implementations

The core FGL procedures consistently involve four steps:

1. Fourier transformation: The target and predicted outputs (error field for regression, network outputs for purification, or signal envelope in optics) are mapped to the frequency domain via FFT or RDFT, applied per relevant dimension (e.g., spatial, temporal).
2. Spectral weight construction: Application-dependent masks/gates—such as exponential decay from DC, bandpass/band-reject windows, or radial binning—are computed to privilege (or suppress) specified frequency regions.
3. Gating and error aggregation: The spectral-domain error or difference is scaled by the weights, often in magnitude, and aggregated using $L_2$ norm or weighted summation.
4. Integration into total objective: The FGL term is combined with the principal loss (e.g., MSE, flow-matching, or physical energy constraints) via a scalar trade-off hyperparameter.

In deep learning, the mask $\mathbf{W}$ can be precomputed based on the expected data dimensions and is usually non-learnable (Tang et al., 5 Jan 2026); in neural operator settings, bin cutoffs and band weights are hyperparameters, allowing practitioners to target frequency-domain weaknesses (such as spectral bias) (Kalimuthu et al., 5 Apr 2025). In photonics, the gating profile $g(\omega,z)$ can be dynamically adapted along the propagation coordinate, tuned to maintain the signal spectrum within the gain window and suppress radiative tails (Peleg et al., 2024).

3. Theoretical Motivation and Domain-Specific Context

Frequency-gated loss functions are motivated by (i) the need for targeted robustness/accuracy in specific frequency ranges and (ii) the recognition of innate biases in standard objectives and architectures:

• Adversarial defense (video/image): High-frequency perturbations dominate adversarial noise. Matching all frequencies equally (as in $L_2$, pixel-space losses) encourages overfitting to these non-semantic artifacts. FGL, by downweighting high-frequency mismatches, enhances semantic (low-frequency) fidelity while suppressing adversarial residuals—demonstrated empirically via PSD analysis (Tang et al., 5 Jan 2026).
• Scientific machine learning (neural operators): Neural networks and Fourier neural operators exhibit "spectral bias," naturally fitting low frequencies first and often failing to recapitulate physically critical mid/high frequencies (e.g., turbulence, sharp gradients). Radially-binned FGL penalizes errors outside the low-frequency band, driving the model to recover more physically faithful spectra (Kalimuthu et al., 5 Apr 2025).
• Waveguide optics: Broadband radiative instability threatens soliton propagation under standard (frequency-independent) gain. Frequency-gated linear gain–loss enables selective suppression of radiative energy that has segregated spectrally (enabled by collisional Raman shift), resulting in long-term, robust soliton stabilization (Peleg et al., 2024).

4. Empirical Benefits and Ablation Evidence

Empirical evaluation has established the efficacy of FGL in multiple domains:

Method Variant	Clean (%)	PGD (%)	CW (%)	DH (%)
Base CFM	94.5	79.0	80.5	14.0
+ Masking	93.0	84.0	86.0	24.0
+ Frequency-Gated Loss	95.5	80.0	81.0	16.0
Masking + FGL (FMVP)	96.0	87.5	89.5	32.0

FGL alone improves clean and robustness metrics, but its effect is synergistic when combined with other structural priors (e.g., masking). FGL outperforms alternative perceptual losses in the context of adaptive attack defense (Tang et al., 5 Jan 2026). In neural operator settings, mid-band and high-band fractional RMSE are reduced by 11.9% and 24.6%, respectively, with spectral energy alignment improved by 75% (mean-energy log-ratio) on challenging turbulent benchmark problems (Kalimuthu et al., 5 Apr 2025). In optical soliton propagation, frequency-dependent gain–loss achieves full suppression of radiative instabilities, maintaining pulse quality, amplitude, and frequency with negligible excursions over extended propagation distances; standard (frequency-independent) gain settings result in rapid pulse breakup (Peleg et al., 2024).

5. Hyperparameter Tuning and Design Considerations

FGL instantiations require careful selection of spectral weights, gating parameters, and loss coefficients to achieve target behavior:

• In adversarial defense, the exponential weight decay $\tau$ is set such that weight at half-Nyquist is $\approx 0.13$ (including floor), with loss coefficient $\lambda_{\rm FGL}=0.2$ balancing spectral and spatial terms (Tang et al., 5 Jan 2026).
• For neural operators, band cutoff indices (e.g., $i_L=4$, $i_H=12$ for $128\times128$ domains) and weighting vector $\mathbf{w}=(0,1,1)$ focus the penalty on mid/high-frequency bands; $\lambda$ in $[0.1,1]$ enables trade-off with pointwise MSE (Kalimuthu et al., 5 Apr 2025).
• In photonic systems, window bandwidth $\Delta\omega$ must cover the instantaneous soliton spectrum; the gate window must track frequency drift (e.g., due to Raman shift). Internal gain $g_0$ should balance loss, and external loss $g_L$ must suffice to suppress out-of-band modes. A plausible implication is that adaptive gating, tracking nonstationary spectral signatures, generalizes this stabilization mechanism (Peleg et al., 2024).

6. Applications and Generalization

Frequency-gated losses are applicable whenever the performance or stability of a model or physical system depends on the selective regulation of error in the frequency domain:

• Adversarial purification: FMVP leverages FGL to suppress high-frequency adversarial noise in video, yielding state-of-the-art accuracy and robust detection against adaptive attacks (Tang et al., 5 Jan 2026).
• Physics-informed regression and operator learning: LoGlo-FNO demonstrates that FGL improves alignment to true physical spectra in PDE surrogates, facilitating accurate recovery of solution fields exhibiting multi-scale phenomena (Kalimuthu et al., 5 Apr 2025).
• Nonlinear optics: Spectral filtering via frequency-dependent gain–loss profiles, interpreted as a form of engineered frequency-gated loss, enables long-distance robust transmission of solitonic pulses against diverse radiative instabilities (Peleg et al., 2024).

While the precise implementation varies, the FGL principle—explicit spectral gating to control the modes dominating application-level metrics—represents an increasingly generalizable paradigm across computational sciences.