Frequency-Informed Masking Overview

• Frequency-informed masking is a method that defines masked units based on spectral features like Fourier coefficients, STFT bins, or octave bands.
• It is employed in self-supervised pre-training and regularization, optimizing tasks such as dereverberation, perceptual masking, and cross-domain adaptation.
• This approach integrates concepts from speech coding, psychophysics, and machine learning to selectively emphasize or attenuate frequency components for improved performance.

Frequency-informed masking denotes a family of masking, corruption, or gain-control procedures in which the masked units are defined with reference to frequency structure rather than only to spatial or temporal position. In the literature considered here, “frequency” may refer to Fourier or STFT coefficients, octave or Bark bands, spectrogram regions, per-pixel spectral signatures, or even token and word occurrence frequency. The resulting methods serve distinct purposes: self-supervised pre-training, supervised regularization, explanation, dereverberation, perceptual masking, acceleration, and cross-domain adaptation (Xie et al., 2022, Kosmopoulou et al., 5 Sep 2025, Brüsch et al., 2024, Doloriel et al., 2024).

1. Conceptual foundations and scope

Frequency-informed masking predates its recent use in self-supervised learning. In speech coding, “full frequency masking by octaves” retains only the single strongest spectral component in each octave and assumes it masks the rest, then reconstructs a denser spectrum by octave-wise beta-distribution filling (Filho et al., 2015). In cochlear-implant dereverberation, time-frequency masking is formulated as a gain matrix $G(t,f)$ applied to a CI-oriented representation, with oracle gains derived from the local speech-to-reverberant ratio and instantiated as an ideal binary mask or ideal ratio mask (Shahidi et al., 2021). In psychophysics, critical band masking measures sensitivity to narrowband perturbations by adding one-octave Gaussian noise at controlled spatial frequencies and estimating the noise level required to reduce recognition to criterion (Subramanian et al., 2023). In STFT-domain speech enhancement, even the window function becomes part of the masking problem, since coefficient modification and inverse reconstruction depend on frame conditioning; this motivated optimization of nearly tight windows for time-frequency masking (Kusano et al., 2018).

Within that broad lineage, recent machine-learning work uses masking not only as a perturbation but as a pretext task, a domain-generalization mechanism, or a compute-allocation policy. The common design question is which frequencies should be removed, attenuated, or emphasized, and at what granularity.

Domain	Frequency variable	Representative masking rule
Vision and hyperspectral SSL	2D Fourier coefficients or spectral FFT over bands	Circular low-/high-pass masking, top-magnitude Com/RCom masks, or per-pixel spectral-frequency masking (Xie et al., 2022, Monsefi et al., 2024, Mohamed et al., 6 May 2025)
Language and VLM pre-training	Token or word occurrence frequency	Rare-token-biased corruption or common-word subsampling (Kosmopoulou et al., 5 Sep 2025, Liang et al., 2024)
Audio and speech SSL	Spectrogram dispersion or full-frequency temporal spans	Dispersion-weighted masking and SpecMask (Niizumi et al., 25 Mar 2026, Makineni et al., 28 Aug 2025)
Detection and restoration	FFT/DCT bands or high-frequency priors	Frequency-band dropout, stochastic annular masking, adaptive sparse processing (Doloriel et al., 2024, Helou et al., 2020, Shang et al., 11 May 2025)
Explanation and perception	Frequency or time-frequency bins, T-F channels	FreqRISE, critical band masking, CI oracle masking (Brüsch et al., 2024, Subramanian et al., 2023, Shahidi et al., 2021)

Two distinctions recur. First, some methods define masking directly in a transformed spectral domain and then invert back to the native input domain, so the backbone still receives an image, waveform, or spectrogram-like tensor. Second, some methods