Frequency-Guided Boundary Refinement Module

• The FGBR module is a neural architecture that leverages frequency-domain decomposition to isolate and enhance high-frequency boundary signals.
• It decouples low- and high-frequency contents to reduce noise and redundant context, enabling precise boundary localization in tasks like action detection and segmentation.
• Empirical results show state-of-the-art gains, with mAP improvements up to 7.6% in temporal action detection and notable increases in segmentation accuracy across multiple domains.

A Frequency-Guided Boundary Refinement (FGBR) module is a neural architectural component that leverages explicit frequency-domain analysis to enhance boundary localization by separating and recombining low- and high-frequency content within learned representations. Instantiated across diverse domains—including temporal action detection, semantic segmentation, and medical image analysis—FGBR’s core mechanism is to distill discriminative, boundary-sensitive signals from feature tensors that would otherwise be dominated by redundant low-frequency context or noise. The module is designed to mitigate the limitations of conventional discriminative backbones, which are typically biased toward low-frequency structures, by introducing specialized frequency decoupling and targeted boundary enhancement (Zhu et al., 1 Apr 2025, Wang et al., 2 Jul 2025, Zhang et al., 12 Dec 2025).

1. Frequency-Domain Motivation and Conceptual Foundations

FGBR modules address a fundamental problem in dense prediction: precise boundary localization requires sensitivity to high-frequency transitions, which standard backbones, pre-trained on natural or highly contextual data, often suppress. These modules employ frequency decomposition—typically via 1D/2D Fourier transforms, temporal difference convolutions, or generative denoising processes—to distinguish between low-frequency (global, semantic/contextual) and high-frequency (local, boundary-dense) content. Learnable mechanisms then amplify or dynamically reweight the high-frequency responses to ensure that action/event or object boundaries are preserved and sharpened, directly countering background interference and smoothing artifacts (Zhu et al., 1 Apr 2025).

2. Core Methodological Architectures

There are several FGBR instantiations, each with domain-specific details:

Temporal Action Detection (FDDet):

• Inputs: Frozen backbone features $X\in\mathbb{R}^{B\times L\times D}$.
• Global Frequency Decoupling (GFD): 1D DFT is applied along the temporal axis. Low frequencies ($k<c$) are preserved, high frequencies are reconstructed as residuals. A scalar $\beta$ adjusts the contribution of high frequency: $X_{\mathrm{dec}} = L(X) + \beta^2 H(X)$, where $L(X)$ is low-pass, $H(X)=X-L(X)$.
• Local High-Frequency Enhancement (LHFE): Sliding windowed convolution over temporal frame differences amplifies rapid local transitions; outputs are fused back.
• Output: Refined features $X_{\mathrm{ref}}$ enriched for action onset/offset transitions (Zhu et al., 1 Apr 2025).

Remote Sensing Segmentation (IDGBR):

• FGBR is realized through a conditional guidance network (derived from Stable Diffusion Unet) and iterative diffusion-denoising, guided by both image and coarse segmentation embeddings. Frequency analysis shows that initial denoising removes global noise (low-$f$), while late stages selectively amplify high-frequency (edge) content, supporting boundary recovery (Wang et al., 2 Jul 2025).

Ultrasound Image Segmentation (FreqDINO):

• High-frequency components at multiple scales are extracted (via MFEA), concatenated, and reduced to a compact “boundary prototype” vector.
• Multi-head cross-modal attention injects the boundary prototype into spatial feature maps, with a fixed scaling ($\omega$), yielding refined predictions that enhance mask–boundary coherence (Zhang et al., 12 Dec 2025).

3. Mathematical Formulation and Data Flow

Temporal Action Detection (FDDet) (Zhu et al., 1 Apr 2025)

• DFT decomposition:

$s_x[k] = \sum_{n=0}^{L-1} x[n] e^{-i 2\pi kn/L}$

• $k<c$0 is recovered by retaining only $k<c$1.
• $k<c$2, $k<c$3.
  • LHFE:

$k<c$4

• Outputs from GFD and LHFE are fused to produce $k<c$5.

Ultrasound Segmentation (FreqDINO) (Zhang et al., 12 Dec 2025)

• Boundary Prototype Distillation:

$k<c$6

• Multi-head Attention:

$k<c$7

$k<c$8

Diffusion-Based Boundary Refinement (IDGBR) (Wang et al., 2 Jul 2025)

• Forward (diffusion): $k<c$9
• Reverse (denoising): $\beta$0
• Frequency-domain filtering analysis demonstrates progressive boundary enhancement in later reverse denoising steps.

4. Integration into Broader Architectures

An FGBR module is typically non-standalone and interfaces as follows:

• Preprocessing: Receives encoder/backbone features (frozen or trainable).
• Boundary Refinement: Applies frequency separation, enhancement, and/or cross-modal boundary injection.
• Output: Refined feature maps forwarded to task-specific heads—TCAR for temporal action detection, boundary/mask decoders for segmentation.
• No explicit frequency-domain loss is imposed in most implementations; rather, task supervision (cross-entropy, Dice, boundary-specific BCE) is applied at final outputs. FGBR itself is trained end-to-end via backpropagation together with the parent model (Zhu et al., 1 Apr 2025, Zhang et al., 12 Dec 2025).

5. Empirical Impact and Ablation Evidence

Temporal Action Detection (THUMOS14, InternVideo2-6B) (Zhu et al., 1 Apr 2025):

• FGAAD only (FGBR): mAP improves from 66.8% (ActionFormer) to 73.6%.
• Full FDDet (FGBR+TCAR): 74.4% mAP, state-of-the-art.
• Best average mAP attained at cutoff $\beta$1; decreasing/increasing $\beta$2 leads to suboptimal results.

Segmentation (BUSI dataset, FreqDINO) (Zhang et al., 12 Dec 2025):

• Adding FGBR to MFEA: Dice improves from 84.17% to 85.13%, mIoU from 74.62% to 76.76%, HD decreases from 44.59 mm to 43.02 mm.

Remote Sensing Semantic Segmentation (IDGBR) (Wang et al., 2 Jul 2025):

• Across DeepLabV3+, SegFormer, DINOv2: weighted F1 (WFm) improvements of +5–13% post-FGBR.
• Gains in WFm are robust across boundary-tolerance thresholds.

6. Implementation Considerations and Hyperparameters

• Temporal Action Detection (FDDet):
  • FFT cutoff: $\beta$3.
  • LHFE: window $\beta$4, kernel size $\beta$5.
  • Optimizer: AdamW, learning rate $\beta$6 (THUMOS14).
• Ultrasound Segmentation:
  • Cross-modal attention: $\beta$7 heads, $\beta$8, $\beta$9 fixed.
  • ReductionNet: two convs and a global pool, final FC to $X_{\mathrm{dec}} = L(X) + \beta^2 H(X)$0-dim vector.
  • Optimizer: Adam, initial LR $X_{\mathrm{dec}} = L(X) + \beta^2 H(X)$1, batch size $X_{\mathrm{dec}} = L(X) + \beta^2 H(X)$2, 300 epochs (Zhang et al., 12 Dec 2025).
• Remote Sensing (IDGBR):
  • Diffusion steps $X_{\mathrm{dec}} = L(X) + \beta^2 H(X)$3 (train), DDIM with $X_{\mathrm{dec}} = L(X) + \beta^2 H(X)$4 (test), $X_{\mathrm{dec}} = L(X) + \beta^2 H(X)$5 (early), batch size $X_{\mathrm{dec}} = L(X) + \beta^2 H(X)$6 (Wang et al., 2 Jul 2025).

7. Theoretical Analysis and Extensions

Analytic results suggest that frequency decomposition aligns with task demands:

• Early denoising in diffusion models suppresses noise at low frequencies, while late-stage restoration selectively amplifies fine edge structures (Wang et al., 2 Jul 2025).
• Supervisor heads that jointly predict boundaries and masks synergistically harness FGBR-refined representations (Zhang et al., 12 Dec 2025).

Adaptive gates (e.g., per-frame $X_{\mathrm{dec}} = L(X) + \beta^2 H(X)$7 in FDDet) are proposed for finer modulation of high-frequency fusion but are not the default (Zhu et al., 1 Apr 2025). Boundary prototype distillation and cross-modal attention (FreqDINO), as well as iterative conditional denoising (IDGBR), represent scalable paradigms for frequency-guided refinement across vision and video modalities.

---

References:

• "FDDet: Frequency-Decoupling for Boundary Refinement in Temporal Action Detection" (Zhu et al., 1 Apr 2025)
• "A Gift from the Integration of Discriminative and Diffusion-based Generative Learning: Boundary Refinement Remote Sensing Semantic Segmentation" (Wang et al., 2 Jul 2025)
• "FreqDINO: Frequency-Guided Adaptation for Generalized Boundary-Aware Ultrasound Image Segmentation" (Zhang et al., 12 Dec 2025)