Frequency-Aware Progressive Debiasing

• Frequency-aware progressive debiasing is a set of strategies that iteratively correct biases from imbalanced frequency content, annotation patterns, and sampling distributions.
• It employs methods like spectral decomposition, adaptive weighting, and progressive objective scheduling to recover overlooked details and balance training dynamics.
• Real-world applications in image segmentation, federated learning, and geophysical inversion demonstrate significant improvements, such as reduced MAE and enhanced spectral fidelity.

Frequency-aware progressive debiasing encompasses a range of algorithmic strategies designed to mitigate systematic errors that arise from imbalances in frequency content, annotation patterns, or sampling distributions during the training of machine learning models. Across domains such as weakly-supervised image segmentation, federated learning, and inverse problems in geophysics, these approaches explicitly incorporate spectral or data-distribution awareness and employ iterative, staged updates to systematically correct bias. Leading implementations include FADeNet in camouflaged object detection (Ge et al., 23 Dec 2025), progressive Sobolev-norm scheduling for spectral control (Yu et al., 2022), counter-based reweighting in federated optimization (Sun et al., 2024), and progressive transfer learning for low-frequency seismic inversion (Hu et al., 2019).

1. Motivations for Frequency-Aware Progressive Debiasing

Supervised learning systems often inherit intrinsic biases from their data or learning dynamics. In weakly supervised segmentation, annotation sparsity induces spatial bias—scribbles are disproportionately concentrated in object centers, leaving boundaries underrepresented and causing models to neglect crucial high-frequency details (Ge et al., 23 Dec 2025). In neural networks, gradient descent under standard losses prioritizes low-frequency (coarse) components, leading to persistence of fine-scale errors (the “spectral bias”) (Yu et al., 2022). Federated learning under nonuniform or temporally correlated sampling yields persistent client participation bias, preventing convergence to the true population optimum (Sun et al., 2024).

Frequency-aware progressive debiasing addresses these limitations by combining explicit spectral decomposition, adaptive weighting, and iteratively refined training objectives. These mechanisms enhance the model’s ability to recover neglected frequencies, spatial regions, or data strata, approaching an unbiased or frequency-neutral solution.

2. Methodologies in Frequency and Bias Modeling

Techniques for frequency-aware debiasing begin with modeling how bias manifests in the training regime:

• Spatial annotation bias in WSCOD: Annotator-generated scribbles yield a supervision mask $y_i$ biased toward the object center. FADeNet targets this via per-pixel weighting $w(i)$, up-regulating poorly annotated regions (Ge et al., 23 Dec 2025).
• Spectral bias in neural nets: Overparameterized networks exhibit a low-frequency bias, as illuminated by neural tangent kernel (NTK) theory—gradient flow decomposes errors into kernel eigenmodes, which decay in a frequency-dependent manner (Yu et al., 2022).
• Distributional bias in FL: Client selection is non-i.i.d., with participation frequencies $\pi_i$ for client $i$ deviating from uniform, thus shifting minimization to $F_\pi(x)$ rather than the true average $F(x)$ (Sun et al., 2024).
• Geophysical regression bias: In FWI, using static synthetic datasets fails to span subsurface variability, leading to underfitting and systematic misrepresentation of low-frequency seismic features (Hu et al., 2019).

Unified by this modeling step, frequency-aware progressive debiasing seeks either to rebalance spectral emphasis (via objective modulation or architecture) or to undo participation or sampling-induced biases via adaptive weighting and iterative correction.

3. Architectural and Algorithmic Implementations

A diversity of implementation strategies exists across modalities:

• FADeNet (D³ETOR) (Ge et al., 23 Dec 2025):
  • Decomposes input images into low-frequency semantic representations using a ViT encoder (LFSE) and high-frequency detail using a Laplacian pyramid (HFDE).
  • Progressively fuses features at multiple scales using windowed cross-attention, enabling joint modeling of global and fine object structure.
  • Employs a learned scribble-probability map for dynamic per-pixel debiasing, directly countering annotation density bias via focal-style loss reweighting.
• Sobolev-Weighted Spectral Learning (Yu et al., 2022):
  • Models network learning in the infinite-width NTK regime with a discrete, density-aware quadrature.
  • Replaces $\mathcal{L}^2$ loss with a Sobolev-norm loss $\|f\|_{H^s}$ allowing the progressive amplification or suppression of different frequency bands by tuning the index $s$.
  • Introduces progressive debiasing schedules—sweeping $s$ from negative (prioritizing low frequencies) to positive (prioritizing high) to recover an unbiased spectral fit.
• Federated Averaging with Debiasing (Sun et al., 2024):
  • Formulates client selection as an Rth-order Markov chain, quantifying the stationary distribution $\pi_R$.
  • Implements running client counters $\lambda^i_t$ to estimate sampling probabilities online, scaling each client’s local update by $\nu^i_t = 1/(N \lambda^i_t)$ for unbiased aggregation—enabling progressive debiasing even under unknown, correlated participation regimes.
• Progressive Transfer Learning in FWI (Hu et al., 2019):
  • Adopts dual data feed DNN architectures to synergistically exploit high-frequency data $d_{HF}$ and beat-tone features $d_{BT}$.
  • Alternates between updating physics-based subsurface models via FWI and retraining/refining the DNN on the enriched synthetic dataset, incrementally reducing bias in low-frequency prediction.

These approaches share a staged or iteratively refined structure, with architectural elements and loss functions targeted at explicit decompositions of frequency or sampling regime.

4. Loss Function Design and Progressive Objective Scheduling

Modifying loss functions is central:

• Per-pixel dynamic weights: In FADeNet, the loss is a combination of segmentation, scribble probability prediction, and debiasing loss at multiple scales. The debiasing term upweights hard-to-annotate or underrepresented spatial regions. The total objective is

$$\mathcal L_{total} = \sum_{k=1}^3 \left[ \mathcal L_{COD}^k + \alpha \mathcal L_{Scrib}^k + \beta \mathcal L_{Debias}^k \right]$$

where the terms are empirically balanced for best performance (e.g., $\alpha = 2$, $\beta=0.5$) (Ge et al., 23 Dec 2025).

• Frequency-weighted losses: Using a frequency-domain weighting operator or Sobolev norm, training can accelerate difficult frequencies or attenuate noise-dominated ones. For Sobolev order $s$, the loss is:

$$L_s(W)=\frac{1}{2}(y-u)^T P_s (y-u), \quad P_s = \sum_k (1+\mu_k)^s A_k A_k^T.$$

Progressive debiasing proceeds by increasing $s$ over training, thus rebalancing spectral emphasis (Yu et al., 2022).

• Federated step scaling: In FL, loss modification is achieved via scaling updates by $\nu_t^i$. Progressive debiasing is implicit, as the counters $\lambda^i_t$ converge to $\pi^i_R$ and corrections to frequency bias accumulate over rounds (Sun et al., 2024).
• Iterative dataset enrichment: In FWI, the loss incorporates frequency-domain weighting and Tikhonov regularization, applied to datasets that are progressively improved by alternating physics-based model updates and DNN re-training (Hu et al., 2019).

5. Empirical Evidence and Benchmark Results

Frequency-aware progressive debiasing demonstrably improves accuracy and mitigates bias in multiple settings:

• FADeNet (D³ETOR): On the COD10K dataset, the full FADeNet module reduces MAE from 0.037 to 0.024 under weak scribble supervision, a $\sim$40% relative improvement. Progressive fusion of frequency channels yields monotonic reductions in error and improved segmentation structure scores ($S_m$), with high-frequency cues especially enhanced in boundary regions (Ge et al., 23 Dec 2025).
• NTK-based spectral debiasing: In controlled regression problems, using a positive $s$ rapidly accelerates convergence of high-frequency modes, while negative $s$ can filter out high-frequency noise; staged scheduling permits recovery of unbiased solutions optimized for the frequency content of the target (Yu et al., 2022).
• Debiased FedAvg: On synthetic regression and MNIST, incorporating debiasing via frequency counters eliminates the asymptotic accuracy gap associated with nonuniform, temporally correlated client participation, matching oracle-uniform benchmarks and outperforming alternatives such as FedVARP (Sun et al., 2024).
• FWI with progressive transfer learning: Shot-by-shot cross-correlation between predicted and true low-frequency data rises from approximately 0.60 to 0.93 over three cycles, and the final velocity models match the ground truth without cycle-skipping artifacts (Hu et al., 2019).

These quantitative and qualitative gains validate the central tenet that progressive, frequency-aware strategies outperform static, frequency-blind or bias-agnostic methods when the training signal is inherently imbalanced.

6. Theoretical Guarantees and Generalization

Rigorous analysis underlines the efficacy and universality of frequency-aware progressive debiasing:

• Spectral learning theory: In the infinite-width NTK regime, frequency-weighted (Sobolev) losses can accelerate, neutralize, or reverse spectral bias. Scheduling $s$ from negative to positive provides staged control over which frequency errors are targeted at each stage (Yu et al., 2022).
• Federated debiasing: Markov chain modeling of participation shows that debiasing FedAvg achieves a $O(1/\sqrt{T})$ convergence rate to the unbiased optimum, with only small mixing-time dependencies, even under unknown and non-stationary availabilities (Sun et al., 2024).
• Progressive transfer: Alternating sample/model enrichment and DNN transfer learning is empirically shown to systematically reduce both underfitting and sampling bias, enabling robust extraction of missing frequency content in nonlinear inverse problems (Hu et al., 2019).

A plausible implication is that such staged debiasing frameworks, customized to the frequency or participation spectrum of the learning system, will generalize to diverse modalities exhibiting non-uniform data or loss surface structure.

7. Cross-domain Applications and Future Research

Frequency-aware progressive debiasing is now represented in diverse domains:

Domain	Method	Exemplary Reference
Image segmentation	FADeNet, cross-attention fusion	(Ge et al., 23 Dec 2025)
Neural net regression	Sobolev loss scheduling (NTK)	(Yu et al., 2022)
Federated learning	Participation debiasing via counters	(Sun et al., 2024)
Geophysical inversion	Progressive dual data feed transfer	(Hu et al., 2019)

Ongoing and future research will likely explore:

• Automated scheduling of debiasing objectives or counter updates tailored to observed frequency deficits.
• Extensions to non-Euclidean domains (e.g., graphs or manifolds) where spectral bias manifests.
• Integration with self-supervised and active learning workflows, where active queries might target maximally biased or underrepresented frequency regions.
• Theoretical bounds in finite-width neural regimes and across broader classes of optimization algorithms.

Emerging trends suggest that frequency-aware progressive debiasing provides a principled foundation for addressing fundamental learning challenges arising from non-uniform sampling, spectral bias, or incomplete supervision across a broad spectrum of machine learning applications.