Frequency-guided Patch Screening (FPS)

• Frequency-guided Patch Screening (FPS) is a learned frequency-domain technique that computes FFT magnitude spectra on overlapping image patches to identify small infrared targets.
• It uses a shallow MLP and geometric-mean fusion to assign pixel-wise target relevance scores, effectively suppressing background noise in low SNR conditions.
• Integrated as the first stage in SEF-DETR, FPS enhances self-attention query initialization, leading to improved detection accuracy and robust performance.

Frequency-guided Patch Screening (FPS) is a learned, frequency-domain module designed to address the challenge of small target discrimination in infrared small target detection (IRSTD) under low signal-to-noise conditions. FPS constructs a high-fidelity, pixel-wise “target relevance” map by analyzing the magnitude spectrum of local image patches via the discrete Fourier transform, with the express goal of suppressing background-dominated features and amplifying areas corresponding to infrared small targets. FPS constitutes the first stage of the SEF-DETR framework, integrating seamlessly with dynamic spatial enhancement and reliability-based fusion mechanisms to guide robust self-attention query initialization and improve end-to-end IRSTD performance (Liu et al., 6 Jan 2026).

1. Mathematical Foundations

Let $I \in \mathbb{R}^{H\times W}$ be a gray-scale infrared image. FPS extracts $J$ overlapping local patches $P_j \in \mathbb{R}^{p\times p}$ (for $j=1,\ldots,J$) using a sliding window of size $p\times p$ and stride $s$, typically with $p$ chosen to fully enclose small targets (e.g., $p=16$ or $32$). For each patch, the discrete 2D Fourier transform is computed:

$$F_j(u,v) = \sum_{x=0}^{p-1} \sum_{y=0}^{p-1} P_j(x,y) \cdot e^{-i2\pi(ux/p + vy/p)}$$

In practice, this transform is computed using FFT, yielding a complex spectrum $J$0. The magnitude spectrum $J$1 is flattened to $J$2. A shallow MLP, followed by a sigmoid layer, produces a scalar target-relevance score $J$3 for each patch:

$J$4

Patch overlap is exploited via geometric-mean fusion: for each pixel $J$5 belonging to $J$6 patches indexed by $J$7,

$J$8

This produces the frequency-guided density map $J$9, in which higher values denote increased likelihood of small target presence based on frequency cues.

Patch-level supervision is provided with binary labels $P_j \in \mathbb{R}^{p\times p}$0 (1 if any ground-truth target pixel lies within $P_j \in \mathbb{R}^{p\times p}$1, otherwise 0), enabling the use of binary cross-entropy loss:

$P_j \in \mathbb{R}^{p\times p}$2

The overall model joint objective is $P_j \in \mathbb{R}^{p\times p}$3, with $P_j \in \mathbb{R}^{p\times p}$4.

2. Algorithmic Procedure

The FPS workflow comprises the following explicit steps:

1. Patch Extraction & FFT: Slide a $P_j \in \mathbb{R}^{p\times p}$5 window to extract $P_j \in \mathbb{R}^{p\times p}$6 overlapping patches from $P_j \in \mathbb{R}^{p\times p}$7. Compute the 2D FFT for each patch to obtain the magnitude spectra.
2. MLP-based Scoring: Flatten each patch’s magnitude spectrum and run it through the MLP+sigmoid to generate $P_j \in \mathbb{R}^{p\times p}$8.
3. Score Fusion: For each pixel $P_j \in \mathbb{R}^{p\times p}$9, fuse the overlapping patch scores using a geometric mean, accumulating in log-space for efficiency.
4. Output: Return $j=1,\ldots,J$0 as a soft, pixel-wise relevance map.

Pseudocode

$p\times p$7

The design favors geometric mean over arithmetic mean to penalize any patch with low $j=1,\ldots,J$1 in overlapping regions.

3. Role within SEF-DETR Architecture

FPS is the initial module in the SEF-DETR pipeline, followed sequentially by Dynamic Embedding Enhancement (DEE) and Reliability-Consistency-aware Fusion (RCF):

• FPS generates $j=1,\ldots,J$2, an initial map of frequency-based target likelihood.
• DEE bilinearly resizes $j=1,\ldots,J$3 to multiple spatial scales matching transformer encoder features $j=1,\ldots,J$4; a binary mask $j=1,\ldots,J$5 is computed via a learned threshold, and used to amplify the encoder features at likely target locations: $j=1,\ldots,J$6.
• RCF applies a lightweight spatial classifier to enhanced features, producing spatial confidence $j=1,\ldots,J$7. Pixel-wise consistency $j=1,\ldots,J$8 and reliability $j=1,\ldots,J$9 are computed based on correspondence between $p\times p$0 and $p\times p$1, yielding final scores $p\times p$2 for top-K query selection.

This positional and frequency-aware initialization feeds forward into the transformer decoder for robust object detection.

4. Empirical Analyses and Visualization

The effectiveness of FPS has been substantiated via ablation studies, frequency-band analyses, and qualitative evaluations on the IRSTD-1k dataset:

• Ablation Study (AP on IRSTD-1k):

| Configuration | AP | |----------------------------|------| | DINO (ResNet-50 baseline) | 37.1 | | + FPS + DEE | 38.3 | | + FPS + RCF | 38.1 | | Full SEF-DETR (FPS+DEE+RCF)| 38.9 |

FPS provides a significant portion of the gain over vanilla detectors.

• Frequency Band Analysis:
  • High-freq only: AP = 37.8
  • Low-freq only: AP = 38.4
  • Full spectrum: AP = 38.9

Inclusion of the full frequency spectrum maximizes performance; mid-band FFT magnitudes, in particular, correlate with true targets.

• Visualizations:
  • FFT magnitude maps highlight mid-frequency spikes for true targets, absent from background/distractor patches.
  • Baseline self-attention queries often trigger on background clutter; FPS-guided queries concentrate on authentic small targets, as shown in spatial confidence maps.
  • End-to-end detection visualizations demonstrate improved suppression of false alarms and recovery of dim or missed targets.

5. Implementation Characteristics and Computational Overhead

FPS introduces modest complexity:

• Parameters: An added MLP with ≈0.27M parameters.
• Computation: +0.08 GFLOPs for a $p\times p$3 input, mainly from $p\times p$4 FFTs of size $p\times p$5.
• Inference Steps: Single MLP forward pass per patch, cumulative geometric-mean fusion via log-domain summation, bilinear interpolation to multiscale maps, and final top-K query selection.
• Training: Joint DETR and FPS loss, with typical balancing ($p\times p$6).

FPS is optimized for efficiency, relying on highly parallelizable FFTs and lightweight per-patch inference. Its geometric-mean fusion is implemented with cumulative log-sums, which do not become a computational bottleneck.

6. Functional Significance and Related Methodologies

FPS serves as a frequency-domain “scout” that leverages local patch FFT magnitudes for soft, adaptive localization of possible small targets. This initial map not only suppresses background-dominated features in downstream encoder outputs (via DEE) but also strengthens reliability in query selection (via RCF). Empirical evaluation conclusively attributes most of SEF-DETR’s gain over vanilla DETR variants to FPS, while imposing minimal computational burden (Liu et al., 6 Jan 2026).

A plausible implication is that frequency-guided screening may become a generalizable paradigm for small object detection in other low SNR conditions where spatial-only cues are insufficient for robust self-attention signal initialization. No controversy or contradiction regarding FPS has been identified in comparative benchmarks.