Frequency-Aware SSL: Methods & Insights

• Frequency-aware SSL is a family of methods that leverage spectral structures (e.g., Fourier, Laplacian) in self-supervision to learn robust features across various domains.
• Key design patterns include frequency-domain corruption with reconstruction, cross-band fusion, and loss shaping to ensure reliable supervision.
• Applications span vision, graphs, audio, and geophysics, demonstrating measurable improvements in classification, segmentation, and signal recovery tasks.

Frequency-aware self-supervised learning denotes a family of self-supervised methods that make frequency structure an explicit part of representation learning. In this literature, “frequency” is not a single object: it may mean Fourier coefficients of images and videos, spatial-frequency variation around edges and textures, Laplacian eigenmodes on graphs, low-frequency seismic bands, pitch contours derived from $F_0$, or 2D time–frequency spectrogram patterns. Across these settings, the common move is to alter masking, augmentation, corruption, reconstruction, or consistency objectives so that the model learns from spectral organization rather than from raw samples alone (Xie et al., 2022, Chen et al., 2022, Li et al., 17 Apr 2026, Gu et al., 24 Jun 2026).

1. Meanings of “frequency” in the literature

The term is used heterogeneously, and its technical meaning is domain-dependent. In vision, it usually refers to Fourier or wavelet components, or to high- and low-frequency regions associated with textures, edges, blur, and illumination. In graphs, frequency is defined through the spectrum of a Laplacian operator. In time series and video, the same idea extends to temporal and spatio-temporal spectra. In MIR, frequency awareness may refer either to explicit pitch trajectories or to 2D spectrogram structure. In seismic inversion, the term often centers on missing low-frequency bands that govern long-wavelength recovery (Jia et al., 2022, Xie et al., 2022, Cheng et al., 2024, Gu et al., 24 Jun 2026).

Domain	Frequency notion	Representative works
Vision	Fourier components, wavelets, spatial high/low frequencies	(Xie et al., 2022, Liu et al., 2022, Liao et al., 27 Aug 2025)
Graph learning	Laplacian spectrum, graph Fourier modes	(Li et al., 17 Apr 2026)
Video and time series	Spatio-temporal frequency or FFT masking	(Kim et al., 2022, Fu et al., 2024)
Audio and music	$F_0$ contours or 2D spectrogram grids	(Noufi et al., 2020, Gu et al., 24 Jun 2026)
Geophysics	Missing low-frequency seismic bands	(Cheng et al., 2024)

This terminological spread is important because it prevents a narrow reading of the field. Frequency-aware SSL is not one method class but a design stance: spectral structure is treated as a source of supervision, bias control, or robustness.

2. Recurrent design patterns

A dominant pattern is frequency-domain corruption followed by reconstruction or inference. MFM masks low- or high-frequency components of an image spectrum and reconstructs only the masked coefficients. FOLK replaces fixed masks with adaptive compression-style Com/RCom selection and adds a teacher branch on original images to reduce the gap between filtered pre-training inputs and natural downstream inputs. FEI applies frequency masking to time series and learns to infer both target embeddings and mask embeddings without using positive–negative pairs. FC-GSSL corrupts nodes and edges with high low-frequency contribution so that the remaining graph input is biased toward high-frequency information (Xie et al., 2022, Monsefi et al., 2024, Fu et al., 2024, Li et al., 17 Apr 2026).

A second pattern is explicit cross-band fusion rather than pure spectral prediction. FSLNet uses parallel frequency and spatial branches so that global information from the Fourier domain and local detail from convolutional processing are fused at each stage. Ge$^2$-AE uses geminated pixel- and frequency-space decoders with reciprocal constraints. The UWF enhancement framework separates deblurring and illumination compensation into frequency-decoupled modules, combining global and local views through an Asymmetric Channel Integration operation and a wavelet-based Color Preservation Unit (Jia et al., 2022, Liu et al., 2022, Liao et al., 27 Aug 2025).

A third pattern is loss shaping: frequency is used to determine which supervision is reliable. In monocular depth estimation, Ambiguity-Masking suppresses boundary pixels that violate photometric correspondence, while a frequency-adaptive Gaussian low-pass filter robustifies photometric loss in high-frequency regions. In self-supervised seismic extrapolation, a data-domain MAE is paired with amplitude-spectrum MAE so that missing-band recovery is directly regularized on real band-limited data (Chen et al., 2022, Cheng et al., 2024).

Denoising work adds a related perspective. I2V is motivated by the observation that pixel-shuffle downsampling weakens spatial noise correlation but can destroy high-frequency details at inference; the abstract reports self-residual learning, an order-variant PD constraint, a noise prior loss, and $\text{PR}^3$ refinement to avoid that texture loss (Lee et al., 2023).

3. Representative mathematical formulations

The field has no single formal core, but several recurring spectral operators are representative.

In graph SSL, frequency is defined by the symmetric normalized Laplacian

$L = I_N - D^{-1/2} A D^{-1/2},$

with eigendecomposition $L = U \Lambda U^T$, so that the graph Fourier transform of node features is

$\hat{X} = U^T X.$

Low eigenvalues correspond to smooth components; high eigenvalues correspond to oscillatory ones (Li et al., 17 Apr 2026).

In visual masked frequency modeling, the target is the masked spectrum itself rather than masked image patches. MFM defines

$$\mathcal{L}_{freq} = \frac{1}{|\Omega_{mask}|}\sum_{(u,v)\in\Omega_{mask}} \left|\mathcal{F}_r(x)(u,v)-\mathcal{F}_o(x)(u,v)\right|^\gamma,$$

where only masked coordinates contribute to the loss, and $\gamma=1$ is reported as the best choice (Xie et al., 2022).

In non-contrastive time-series representation learning, FEI uses dual inference objectives: $$\mathcal{L} = \frac{1}{N}\sum_{i=1}^{N}\big\|u'_i-\hat{u}'_i\big\|_2^2 + \big\|m_i-\hat{m}_i\big\|_2^2,$$ with one branch inferring the embedding of a frequency-masked target series and the other inferring the mask embedding itself (Fu et al., 2024).

In seismic low-frequency extrapolation, the total objective combines signal-space and spectrum-space consistency: $F_0$0 where $F_0$1 is MAE in the data domain and $F_0$2 is MAE on amplitude spectra (Cheng et al., 2024).

These formulations differ in the object being predicted—complex coefficients, embeddings, graph modes, or extrapolated low bands—but they share a common logic: the pretext target is spectrally structured, and that structure constrains what the representation must preserve.

4. Vision and imaging applications

Visual pre-training is the most densely developed branch of the area. MFM shows that a non-Siamese frequency-domain mask-and-predict objective can reach 83.1 top-1 on ImageNet-1K with ViT-B/16 and 48.6 mIoU on ADE20K, while also improving robustness benchmarks such as FGSM, PGD, IN-C, IN-A, IN-R, and IN-Sketch (Xie et al., 2022). FOLK addresses the two specific weaknesses of fixed spectral masks and filtered-image domain gap, reaching 81.6 top-1 at 300 pre-training epochs and 82.1 at 800, with strong few-shot behavior under 10% and 1% ImageNet-1K labels (Monsefi et al., 2024). Ge$F_0$3-AE couples pixel and frequency reconstruction and reports 84.8 top-1 for ViT-B, 86.6 for ViT-L, and 87.7 for ViT-H, together with 48.9 mIoU on ADE20K for ViT-B (Liu et al., 2022).

Dense prediction and geometry estimation extend the same logic to task-specific supervision. FSLNet-L, which jointly learns in frequency and spatial domains, yields KITTI AbsRel $F_0$4 and Cityscapes mIoU $F_0$5 without pretraining, while using approximately $F_0$6M parameters (Jia et al., 2022). In self-supervised monocular depth estimation, the combined use of Auto-Blur and Ambiguity-Masking improves a ResNet-18 base model on KITTI from AbsRel $F_0$7 to $F_0$8, with SqRel $F_0$9, RMSE $^2$0, and RMSE$^2$1 $^2$2 (Chen et al., 2022).

Restoration problems provide another major application. In real-image denoising, I2V is explicitly motivated by the frequency-destructive effect of PD/AP inference; the abstract states that it outperforms state-of-the-art self-supervised blind denoising methods, including several supervised methods, in PSNR, SSIM, LPIPS, and DISTS on real-world sRGB images (Lee et al., 2023). In UWF retinal image enhancement, the combination of frequency-decoupled deblurring and Retinex-guided illumination compensation achieves NIQE $^2$3, BRISQUE $^2$4, and PIQE $^2$5 on 434 test images, and raises diabetic-retinopathy grading from ACC $^2$6 and F1 $^2$7 on original images to ACC $^2$8 and F1 $^2$9 (Liao et al., 27 Aug 2025).

5. Extensions beyond vision

In video SSL, FreqAug removes selected spatio-temporal frequency components so that representations emphasize dynamic cues rather than static information; the abstract reports consistent improvements on five action-recognition and two temporal-action-localization tasks (Kim et al., 2022). In time series, FEI replaces contrastive pair construction with prompted embedding inference and reports an average linear-evaluation accuracy of $\text{PR}^3$0 and an end-to-end average of $\text{PR}^3$1, together with regression results of $\text{PR}^3$2 MSE/MAE on C-MAPSS and $\text{PR}^3$3 on Bearing (Fu et al., 2024).

Music applications divide into two lines. One line treats frequency explicitly as pitch. Context-aware $\text{PR}^3$4-based SSL improves VocalSet singer classification from $\text{PR}^3$5 to $\text{PR}^3$6 and RAVDESS emotion classification from $\text{PR}^3$7 to $\text{PR}^3$8, with downstream gains of as much as $\text{PR}^3$9 over traditional statistical contour features (Noufi et al., 2020). The other line treats music as a 2D time–frequency grid. PupuJEPA trains JEPA directly on spectrogram patches and, on MARBLE, a Large model reaches $L = I_N - D^{-1/2} A D^{-1/2},$0 GS key refined accuracy, $L = I_N - D^{-1/2} A D^{-1/2},$1 GTZAN genre accuracy, and up to $L = I_N - D^{-1/2} A D^{-1/2},$2 F1 on GTZAN beat tracking, while attention maps show repeated rhythmic events and harmonic relations (Gu et al., 24 Jun 2026).

Graph SSL adopts a different spectral formalism but the same self-supervised principle. FC-GSSL builds high-frequency-biased corrupted graphs from low-frequency contribution scores and supervises reconstruction of both general node features and low-frequency structural features. It reports state-of-the-art or best mean accuracy on all six node-classification benchmarks, including $L = I_N - D^{-1/2} A D^{-1/2},$3 on BlogCatalog, $L = I_N - D^{-1/2} A D^{-1/2},$4 on Chameleon, $L = I_N - D^{-1/2} A D^{-1/2},$5 on Squirrel, $L = I_N - D^{-1/2} A D^{-1/2},$6 on Actor, $L = I_N - D^{-1/2} A D^{-1/2},$7 on arXiv-year, and $L = I_N - D^{-1/2} A D^{-1/2},$8 on Penn94, while also improving molecular prediction and QM9 transfer (Li et al., 17 Apr 2026).

In geophysics, frequency-aware SSL is centered on the recovery of missing low bands rather than detail enhancement. Self-supervised seismic low-frequency extrapolation is trained directly on real band-limited data, recovers missing components down to about $L = I_N - D^{-1/2} A D^{-1/2},$9 Hz in a synthetic cutoff-5 Hz setting and about $L = U \Lambda U^T$0 Hz on field data, and improves full waveform inversion by mitigating cycle-skipping (Cheng et al., 2024).

6. Limitations, misconceptions, and evolving directions

A common misconception is that frequency-aware learning always means amplifying high-frequency detail. The literature is more heterogeneous. Some methods intentionally suppress or blur high-frequency regions to make supervision fairer, as in frequency-adaptive photometric loss for depth estimation. Others recover missing low frequencies because long-wavelength content is the critical failure mode, as in seismic extrapolation. FC-GSSL goes further by corrupting low-frequency contributors to create high-frequency-biased inputs, yet reconstructing low-frequency and general targets so that multi-band fusion is enforced (Chen et al., 2022, Cheng et al., 2024, Li et al., 17 Apr 2026).

Another recurring result is that frequency-only processing is usually insufficient. In dense prediction, CNN-only local learning outperforms LFL-only global learning, while the parallel CNN+LFL fusion is best. In Ge$L = U \Lambda U^T$1-AE, frequency-only reconstruction is inferior to the geminated pixel–frequency design. In UWF enhancement, removing either ACI or CPU degrades both no-reference image quality metrics and diabetic-retinopathy grading performance (Jia et al., 2022, Liu et al., 2022, Liao et al., 27 Aug 2025). This suggests that successful methods generally combine spectral structure with spatial, temporal, or graph-local context rather than replacing one with the other.

The phrase itself is also not semantically uniform. In long-tailed SSL, FASSL uses “frequency-aware” to denote latent class-frequency awareness rather than Fourier or Laplacian frequency: prototypes are learned from unlabeled long-tailed data, and instance weights inversely proportional to aggregated prototype similarity emphasize rare-class samples, yielding $L = U \Lambda U^T$2 on CIFAR100-LT and $L = U \Lambda U^T$3 on CIFAR10-LT under linear evaluation (Lin et al., 2023). Across the broader literature, open issues include FFT/iFFT and eigendecomposition overhead, instability in target–context designs, sensitivity to masking policy, and residual domain gaps between filtered pre-training inputs and natural downstream data; these concerns are explicit in FSLNet, FC-GSSL, PupuJEPA, and FOLK (Jia et al., 2022, Li et al., 17 Apr 2026, Gu et al., 24 Jun 2026, Monsefi et al., 2024).

Taken together, the field has moved from fixed hand-crafted spectral corruption toward adaptive masking, cross-domain fusion, and task-specific supervision. A plausible implication is that future frequency-aware self-supervised learning will be defined less by any single transform than by whether the pretext task exposes the spectral structure that is actually causal for downstream generalization.