Median-Frequency Feature Fusion (MFFF)

• MFFF is a neural network module that enhances small-object detection by fusing robust median pooling with selective frequency-domain attention.
• It employs a dual-branch design—one branch computes global channel statistics via median pooling and MLP, while the other leverages FFT-based frequency weighting.
• Empirical results show improved detection metrics in UAV imagery, with mAP gains up to 1.6%, highlighting its impact on challenging visual tasks.

Median-Frequency Feature Fusion (MFFF) is a neural network module designed to improve small-object detection in complex visual environments, notably in UAV (unmanned aerial vehicle) imagery. The MFFF module remedies two central obstacles: statistical suppression of small-object features by dominant background activations and the inadequate amplification of high-frequency edge and texture cues critical for recognizing tiny targets. MFFF achieves robust, discriminative feature fusion by combining a median-stabilized channel-attention branch and a frequency-domain attention branch, yielding improved detection accuracy and contextual sensitivity, particularly for instances with spatial footprints below 32×32 pixels (Huo et al., 30 Oct 2025).

1. Motivation and Theoretical Background

Small-object detection in UAV imagery is challenged by:

• The overwhelming majority of pixels representing background or large objects, which bias global pooling operators (average or max) towards extreme or non-representative activations.
• The masking of subtle object cues from minor instances, whose feature contributions are drowned out by few bright or outlier activations.
• The essential role of high-frequency spectral information—edges, outlines, fine textures—which encode salient signals of tiny targets but are often lost or attenuated by conventional 2D convolutions.

MFFF addresses these limitations by introducing:

• Global Median Pooling (GMP) as a third global statistic alongside average and max pooling, generating a robust estimator less sensitive to outliers.
• Frequency-Domain Attention by explicitly transforming features with a 2D Fast Fourier Transform (FFT), applying learned attention weights over spectral components, and reconstructing the result with an inverse FFT (IFFT). This allows selective amplification of frequency bands that characterize small-object structure.

By fusing spatial-domain (median-aware) and frequency-domain (spectral selective) attention, MFFF forms a composite, differentiable reweighting mechanism for feature maps (Huo et al., 30 Oct 2025).

2. Mathematical Formulation

Given an input tensor $X \in \mathbb{R}^{C \times H \times W}$:

• Global Average Pooling (GAP): $$a_c = \frac{1}{HW} \sum_{i=1}^{H} \sum_{j=1}^{W} X_{c,i,j}$$
• Global Max Pooling (GMPₘₐₓ): $p_c = \max_{i,j} X_{c,i,j}$
• Global Median Pooling (GMPₘₑd): $m_c = \text{median}_{i,j} X_{c,i,j}$

Channel-attention branch (DCAM):

1. $P = a + p + m$
2. $s = \sigma(W_2 \, \text{ReLU}(W_1 P))$
   • $W_1 \in \mathbb{R}^{(C/r) \times C}$, $W_2 \in \mathbb{R}^{C \times (C/r)}$, $r$ is the reduction ratio (default: 16), $\sigma$ is the element-wise sigmoid.
3. $$a_c = \frac{1}{HW} \sum_{i=1}^{H} \sum_{j=1}^{W} X_{c,i,j}$$0

Frequency-attention branch (FSAM):

1. $$a_c = \frac{1}{HW} \sum_{i=1}^{H} \sum_{j=1}^{W} X_{c,i,j}$$1
2. $$a_c = \frac{1}{HW} \sum_{i=1}^{H} \sum_{j=1}^{W} X_{c,i,j}$$2
3. $$a_c = \frac{1}{HW} \sum_{i=1}^{H} \sum_{j=1}^{W} X_{c,i,j}$$3
   • FreqConv is a learned complex linear mapping, realized as $$a_c = \frac{1}{HW} \sum_{i=1}^{H} \sum_{j=1}^{W} X_{c,i,j}$$4 convolutions across real and imaginary channels.
4. $$a_c = \frac{1}{HW} \sum_{i=1}^{H} \sum_{j=1}^{W} X_{c,i,j}$$5
5. $$a_c = \frac{1}{HW} \sum_{i=1}^{H} \sum_{j=1}^{W} X_{c,i,j}$$6

Fusion and Output:

1. $$a_c = \frac{1}{HW} \sum_{i=1}^{H} \sum_{j=1}^{W} X_{c,i,j}$$7
2. $$a_c = \frac{1}{HW} \sum_{i=1}^{H} \sum_{j=1}^{W} X_{c,i,j}$$8

3. Module Architecture and Forward Pass

MFFF operates as follows:

1. Receives input $$a_c = \frac{1}{HW} \sum_{i=1}^{H} \sum_{j=1}^{W} X_{c,i,j}$$9, typically multi-scale features fused after SPDConv.
2. Split: Simultaneously processes $p_c = \max_{i,j} X_{c,i,j}$0 through DCAM and FSAM branches.
3. DCAM: Computes channel statistics, sums, passes through a 2-layer MLP ($p_c = \max_{i,j} X_{c,i,j}$1 conv → ReLU → $p_c = \max_{i,j} X_{c,i,j}$2 conv → sigmoid), then broadcasts channel weights back onto $p_c = \max_{i,j} X_{c,i,j}$3 by elementwise multiplication.
4. FSAM: Applies $p_c = \max_{i,j} X_{c,i,j}$4 conv; 2D FFT; learns frequency-domain attention (as $p_c = \max_{i,j} X_{c,i,j}$5 real-valued convolutions on real/imaginary components); processes IFFT; further $p_c = \max_{i,j} X_{c,i,j}$6 conv refines the branch output.
5. Fusion: The branch results are summed, projected to a single $p_c = \max_{i,j} X_{c,i,j}$7 attention map via $p_c = \max_{i,j} X_{c,i,j}$8 conv and sigmoid.
6. Output: The map $p_c = \max_{i,j} X_{c,i,j}$9 reweights the original $m_c = \text{median}_{i,j} X_{c,i,j}$0 by channel and position, producing $m_c = \text{median}_{i,j} X_{c,i,j}$1 for subsequent processing.

Pseudocode for the forward computation is given below: $m_c = \text{median}_{i,j} X_{c,i,j}$7 (Huo et al., 30 Oct 2025)

4. Placement Within Detection Frameworks

MFFF is implemented within the PT-DETR object detection pipeline as part of the Multi-Scale Feature Refinement Pyramid, specifically after the SPDConv step that restores resolution for low-level (P2) features. At this point, multi-scale feature maps (P2–P5) are aggregated. The MFFF module replaces the Feature Pyramid Network's final output, delivering median-and-spectral-attended features to downstream hybrid encoder (AIFI/CCFM) and deformable DETR decoder components (Huo et al., 30 Oct 2025).

5. Empirical Results and Performance Contribution

Ablation studies on the VisDrone2019 dataset reveal the incremental effects of the Multi-Scale Feature Refinement Pyramid (SPDConv + MFFF):

• mAP₅₀ improved from 36.8% to 37.6% (+0.8%)
• mAP₅₀₋₉₅ improved from 26.4% to 27.6% (+1.2%)

Since SPDConv alone refines spatial downsampling for P2, MFFF's unique impact is attributed to (a) outlier-robust channel statistics via median pooling and (b) selective frequency-band enhancement through FFT-attended weighting. When used in combination with PADF and Focaler-SIoU within PT-DETR:

• mAP₅₀ reaches 38.4% (+1.6% over RT-DETR)
• mAP₅₀₋₉₅ reaches 28.1% (+1.7%) (Huo et al., 30 Oct 2025)

This indicates a measurable impact on sensitivity to small-object boundaries and contextual detail.

6. Training Hyperparameters and Implementation Details

Key hyperparameters for MFFF within PT-DETR are as follows:

• Reduction ratio ($m_c = \text{median}_{i,j} X_{c,i,j}$2): Default 16 in the DCAM branch MLP.
• Frequency-domain conv kernels: $m_c = \text{median}_{i,j} X_{c,i,j}$3 convolution, no additional frequency binning; operates over full FFT grid for both real and imaginary components.
• Learning rate: $m_c = \text{median}_{i,j} X_{c,i,j}$4
• Optimizer: Adam ($m_c = \text{median}_{i,j} X_{c,i,j}$5, weight_decay = $m_c = \text{median}_{i,j} X_{c,i,j}$6)
• Batch size: 4
• Input image size: 640×640
• Epochs: 300
• No specialized training schedule: Uses standard cosine decay; MFFF parameters are trained jointly with the network under the same loss functions (classification + Focaler-SIoU).

7. Context and Significance in Visual Recognition

MFFF bridges spatial-robustness with spectral-selectivity by uniting statistical median-pooling and frequency-specific attention in a lightweight and differentiable design. It addresses domain-specific weaknesses in global-pooling statistics and convolutional inability to exploit informative frequency bands for small-object detection. Its successful application in UAV scenarios—where objects are often occluded, minute, or embedded in clutter—demonstrates its utility for future research in low-SNR detection pipelines, high-resolution segmentation, and scenarios with non-uniform object scale distributions (Huo et al., 30 Oct 2025). A plausible implication is that analogous median-frequency fusion strategies may hold promise wherever feature distributions are heavily skewed or frequency signatures are central to discriminative recognition.