Frequency-Domain FFDPM: Fine-Grained Degradation Analysis

• FFDPM is a neural module that uses frequency transforms to explicitly characterize and disentangle image degradations with HVS priors.
• It integrates spectral analysis techniques like Degradation Frequency Curve and spectral tokenization to support restoration, quality assessment, and multi-modal fusion.
• Empirical results show improved PSNR, SSIM, and perceptual metrics, demonstrating robustness against mixed and unseen degradations.

A Frequency-domain Fine-Grained Degradation Perception Module (FFDPM) is a neural module that leverages frequency-domain transforms to explicitly characterize and localize image degradations, enabling fine-grained, perceptually-aligned restoration, assessment, or fusion. FFDPM manifests in several representative designs centered around spectral analysis, contrast sensitivity, data-driven frequency tokenization, and cross-modal or task-oriented conditioning, as evidenced in all-in-one restoration, AI-generated image quality assessment, and robust image fusion. It aims to disentangle degradation factors across frequency bands, often guided by human visual system (HVS) priors, and to propagate frequency-aware cues within modern architectures.

1. Motivation and Conceptual Foundations

The core motivation behind FFDPM is the recognition that natural and synthetic image degradations (e.g., noise, haze, rain, artifacts from AGI, or modality-specific degradations in fusion) induce characteristic, often distinctive, patterns in the frequency domain. Conventional restoration or assessment pipelines typically absorb these degradations as hidden factors, resulting in entangled feature spaces, poor discrimination of unseen degradations, and sub-optimal generalization in mixed degradation regimes.

Frequency-domain analysis provides an interpretable, content-robust coordinate space for quantifying these effects. By explicitly modeling degradation responses over spectral bands—rather than resorting to hard-to-interpret or purely spatial features—FFDPM exposes and operationalizes degradation, creating reusable priors for downstream restoration, perceptual assessment, or cross-modal fusion tasks (Huang et al., 17 May 2026, Li et al., 14 Jul 2025).

2. Theoretical Frameworks and Mathematical Formulations

FFDPM encompasses several mathematically grounded instances:

• Degradation Frequency Curve (DFC): Given a degraded image $y$ and its clean counterpart $x$, form the residual $r = y - x$, then compute the 2D Fourier transform $\mathcal F(\cdot)$ on both. The frequency spectrum is partitioned into $B$ rings via Gaussian masks $\mathcal M_b$ centered at frequencies $\mu_b$ with widths $\sigma_b$, yielding energy ratios:

$$\tilde R_b = \frac{\sum_{h,w}\mathcal M_b(h,w)|\mathcal F(r)(h,w)|^2}{\sum_{h,w}\mathcal M_b(h,w)|\mathcal F(y)(h,w)|^2}$$

Normalize to get $R_b$, and the DFC becomes $x$0 (Huang et al., 17 May 2026).

• Spectral Tokenization: The DFC vector is decomposed into $x$1 "band-wise spectral tokens" $x$2 using equal-area partitioning of the curve and adaptive refinement via softmax-weighted candidates, with each token representing localized spectral priors across scales.
• HVS-Aligned Perception: Taking image crops, the magnitude spectrum $x$3 is extracted via FFT, and a contrast sensitivity function (CSF), $x$4, weights frequency bands to align response with human perceptual sensitivity. A patch-level "HVS weight" $x$5 is obtained via a sigmoid of total CSF-weighted energy (Li et al., 14 Jul 2025).
• Joint Spatial-Frequency Modulation: For task-specific degradation (e.g., haze), pixel-wise confidence maps (from the dark channel prior) modulate features spatially, and compact Fourier-domain MLPs apply learned, per-frequency transformations, fusing spatial and spectral cues (Zheng et al., 15 Jul 2025).

3. Architectural Instantiations

FFDPM is realized in deep architectures through several orthogonal but compatible strategies:

• Token-Conditioned Restorers: In DFC-IR, an encoder builds a feature pyramid; the decoder at each scale estimates the DFC, extracts band-wise spectral tokens, applies band-wise cross-attention modulation, and adaptively aggregates via learned weights. Global structural context is preserved through additional self- and cross-attention among aggregated and contextual features. Losses combine multi-scale $x$6 image supervision and frequency-consistency (spectral) terms (Huang et al., 17 May 2026).
• HVS-Guided Perceptual Assessment: In SC-AGIQA, patches are transformed to frequency space, weighted by CSF, then fused with data-driven spatial/channel importance for a vectorized quality indicator (VQI), appended with semantic consistency indicators for final judgment (Li et al., 14 Jul 2025).
• Frequency-Driven Fusion and Dehazing: Frequency-domain FFDPM modules are embedded in dual-domain fusion (via sub-band decomposition, prompt-guided affine gating, and transformer-based enhancement for modality-specific features (Zhang et al., 5 Sep 2025)) and in haze estimation (dark channel–derived spatial attention, learnable frequency MLP modulation, and tight fusion with multi-level gating for detail recovery (Zheng et al., 15 Jul 2025)).

Paper & System	FFDPM Purpose	Frequency Analysis Method
(Huang et al., 17 May 2026) DFC-IR	Restoration/Generalization	Gaussian ring DFC, tokenization
(Li et al., 14 Jul 2025) SC-AGIQA	Fine-grained AGI assessment	Patchwise FFT + CSF weighting
(Zhang et al., 5 Sep 2025) GD²Fusion	Modality-specific fusion	DWT subbands + prompt gating
(Zheng et al., 15 Jul 2025) DGFDNet (HAFM)	Haze enhancement	FFT+MLP modulated by dark-channel map

4. Task-Specific Adaptations

FFDPM variants are tailored for distinct downstream tasks:

• All-in-One Blind Restoration: DFC-based FFDPMs enable visuo-spectral separation of compound/mixed degradations, supporting restoration under both seen and unseen corruptions by disentangling them in the degradation coordinate space (Huang et al., 17 May 2026).
• AI-Generated Image Quality Assessment: SC-AGIQA’s FFDPM boosts perceptual distortion sensitivity, especially for fine-grained artifacts in AGI, via HVS-aligned frequency weighting combined with learned attentional pooling (Li et al., 14 Jul 2025).
• Multi-Source Fusion and Dehazing: For fusion of infrared/visible streams, FFDPM submodules extract and suppress degradation in each frequency band, while for dehazing, FFDPM (as HAFM) adapts the learned frequency response by spatial haze confidence for both global spectral and local fine-structure enhancement (Zhang et al., 5 Sep 2025, Zheng et al., 15 Jul 2025).

5. Training and Loss Formulations

Training strategies for FFDPM-centered systems integrate multi-objective criteria:

• Restoration Losses: Multi-scale image-level $x$7 losses supervise at all scales, while spectral alignment is enforced by $x$8 loss between output and ground-truth spectra. The total loss is $x$9, where typically $r = y - x$0, $r = y - x$1 (Huang et al., 17 May 2026).
• Perceptual Quality Losses: For AGI assessment, end-to-end training uses Smooth $r = y - x$2 loss against Mean Opinion Score (MOS), with VQI as the FFDPM-induced quality proxy (Li et al., 14 Jul 2025).
• Fusion/Dehazing Losses: Combined intensity, texture, and chrominance losses with empirically set weights ($r = y - x$3, $r = y - x$4, $r = y - x$5) ensure fidelity to source modality references and robust preservation of image structure and color (Zhang et al., 5 Sep 2025, Zheng et al., 15 Jul 2025).

6. Empirical Evidence and Comparative Impact

FFDPM modules demonstrably enhance performance over ablated or naïve designs:

• Explicit frequency modeling (DFC) yields robust disentanglement and quantification of degradation under mixed and previously unseen conditions, translating to state-of-the-art PSNR, SSIM, LPIPS/FID, and generalization (Huang et al., 17 May 2026).
• HVS-weighted frequency pooling in FFDPM (SC-AGIQA) improves Spearman correlation by 0.6–1.0 points over naive pooling in AGIQA benchmarks, confirming superior sensitivity to subtle, localized artifacts (Li et al., 14 Jul 2025).
• For haze removal, HAFM (FFDPM) alone achieves PSNR=39.35 dB, SSIM=0.994; removing its frequency or spatial modulation results in ≈1 dB loss, and replacing its dark channel spatial prior reduces PSNR by 0.40 dB (Zheng et al., 15 Jul 2025). Qualitative maps show progressive focus on true haze-dense regions and relevant bands.

7. Summary of Variants and Unifying Features

FFDPMs across these domains share several hallmarks:

• Local-global spectral coupling: Local frequency responses are extracted (patch, ring, or subband) and globally integrated, sometimes via tokenization and adaptive attention or pooling.
• Human visual system priors: Classical psychophysical CSFs or perceptually-inspired weighting increase the alignment with subjective quality assessments and restoration perceptiveness.
• Task-conditional modulation: Incorporation of auxiliary priors (e.g., prompt embeddings, dark channel features) enables FFDPMs to adapt frequency sensitivity in context-dependent and data-driven manners.
• Analytic flexibility: FFDPM design accommodates band-wise, token-wise, or subband-specific processing; can be inserted in restoration, assessment, fusion, or enhancement pipelines with minimal architectural assumptions.

These modules transition from implicit, ill-posed degradation handling to an explicit, measurable, and reusable spectral prior mechanism, consistently improving discriminability, interpretability, and generalization across scientifically rigorous image processing benchmarks (Huang et al., 17 May 2026, Li et al., 14 Jul 2025, Zhang et al., 5 Sep 2025, Zheng et al., 15 Jul 2025).