FSDC-DETR: A Frequency-Spatial Domain Collaborative DETR for Small Object Detection

Abstract: Small object detection (SOD) remains a challenging task in real-world applications. Despite recent advances, existing detectors remain limited by rigid processing that entangle spatial aggregation with implicit frequency aliasing and truncation, leading to inadequate preservation of high-frequency components for SOD. To tackle these limitations, we propose a Frequency-Spatial Domain Collaborative Detection Transformer (FSDC-DETR), a novel collaborative framework that explicitly models complementary spatial and frequency representations. Specifically, we first introduce Dual-Branch Frequency-Spatial Adaptive Fusion (DBFSAF) to enhance frequency diversity and adaptively capture frequency-spatial domain discriminative representations. Building on these representations, a frequency-spatial interaction scheme is further explored within the hybrid encoder to enable progressive feature propagation to the decoder. In particular, structure-aware frequency-spatial aggregation is achieved through Shunt Frequency-Spatial Feature Fusion (SFS-FF), establishing bidirectional interaction and progressive cross-scale propagation between frequency and spatial representations for coherent discriminative modeling. Meanwhile, informative high-frequency responses are preserved during scale transitions through Frequency-Spatial Dynamic Downsampling (FSD-Down), thereby minimizing frequency degradation throughout multi-scale fusion for the precise SOD. Experimental results demonstrate that FSDC-DETR achieves state-of-the-art performance, improving AP by 6.4 on VisDrone-DET2019 and 6.6 on AITODv2, with gains of 6.8 and 6.9 AP for small objects. The code is available at github.com/nevereverinsomnia/FSDC-DETR.

• The paper introduces a collaborative DETR that preserves high-frequency details using dual-branch fusion methods like DBFSAF, SFS-FF, and FSD-Down.
• It employs adaptive frequency-spatial processing with dynamic convolutions and learnable wavelet transforms to enhance small object localization.
• Empirical results on VisDrone-DET2019 and AITODv2 validate significant performance gains, confirming the framework's superiority over existing models.

FSDC-DETR: Frequency-Spatial Domain Collaborative Detection Transformer for Small Object Detection

Introduction and Motivation

Small object detection (SOD) in real-world imagery is impeded by limitations in current feature aggregation paradigms, which frequently entangle spatial and frequency processing, causing loss of high-frequency cues central to precise localization. CNN and ViT architectures offer distinct spectral properties: CNNs preserve high-frequency details via local receptive fields, while ViTs deliver strong global semantic context but demonstrate pronounced low-pass filtering that suppresses fine structures. Hybrid designs, such as DEIMv2, concatenate these branches but lack spectral alignment, resulting in aliasing and truncated high-frequency propagation, which collectively degrade SOD performance.

Figure 1: Frequency response analysis for CNN, ViT, DEIMv2, and FSDC-DETR backbones, highlighting FSDC-DETR's superior maintenance of high-frequency components for SOD.

FSDC-DETR introduces a principled, frequency-spatial collaborative pipeline to construct, propagate, and preserve frequency-aware representations across all feature hierarchy stages. By employing explicit spectral techniques—including dynamic convolution, learnable wavelet transforms, and bidirectional frequency-spatial fusion—the framework augments high-frequency signal preservation and enhances local structure modeling while maintaining robust global context.

Overall Architecture

FSDC-DETR architects a three-stage frequency-spatial collaborative backbone and hybrid encoder-decoder DETR architecture. The pipeline integrates:

• Dual-Branch Frequency-Spatial Adaptive Fusion (DBFSAF) for explicit frequency-diverse feature construction
• Shunt Frequency-Spatial Feature Fusion (SFS-FF) in a hybrid encoder for cross-scale, spectral-coherent propagation
• Frequency-Spatial Dynamic Downsampling (FSD-Down) for preserving spectral integrity during resolution transitions

  Figure 2: Overall architecture of FSDC-DETR, showing dual-branch backbone, frequency-adaptive fusion, frequency-spatial encoders, and decoders for robust SOD.

This explicit separation and controlled interaction of spatial and frequency pathways prevent excessive suppression of discriminative high-frequency content, which is essential for small instance localization.

Dual-Branch Frequency-Spatial Adaptive Fusion (DBFSAF)

The DBFSAF block orchestrates a coordinated fusion of ViT and CNN representations by stacking frequency-dynamic convolution (FDConv) atop concatenated backbone features, partitioning channels with a learnable partial ratio $\gamma$ to enhance frequency diversity and suppress inter-channel redundancy. A composite module—combining multi-kernel spatial refine units and frequency processing units—jointly refines local structure and frequency responses with minimal parameter overhead.

Figure 3: Schematic of the DBFSAF module, showing dynamic frequency diversification and partial frequency-spatial refinement.

Exploiting the complementary frequency behaviors of ViT (low-frequency) and CNN (high-frequency) ensures that fine-grained boundary details necessary for SOD are emphasized without sacrificing global semantics, as substantiated by the observed improvements in frequency response (Figure 1).

Shunt Frequency-Spatial Feature Fusion (SFS-FF)

SFS-FF is positioned within the hybrid encoder to further reinforce frequency-coherent aggregation across scales. The module applies 1×1 convolutions to decompose features into spatial refinement ($F_S$), frequency refinement ($F_F$), and a residual connection, then applies fast Fourier transforms (FFT/IFFT) and spatial refinement modules for per-path specialization.

Figure 4: SFS-FF module showing the shunt and recombination of frequency and spatial features for structure-aware multi-scale aggregation.

By explicitly modeling cross-pathway interactions in the spectral and spatial domains, SFS-FF supports robust, progressive feature enrichment—mitigating the degradation of high-frequency cues critical for SOD through deep stacking and multiscale operations.

Frequency-Spatial Dynamic Downsampling (FSD-Down)

The FSD-Down module is a hybrid downsampling operator that leverages learnable wavelet (DWT) decomposition and grouped convolutions, preserving sub-band spectral information (LL, LH, HL, HH) during resolution reduction. Unlike conventional pooling, which acts as a low-pass filter and erodes fine structure, this module adaptively calibrates sub-bands and reweighs outputs with channel attention to best preserve high-frequency content.

Figure 5: FSD-Down operation using DWT and grouped convolutions to enable spatial downsampling with minimal frequency loss.

The hybridization of discrete wavelet transforms and learnable convolutions ensures downsampled features retain both spectrally informative and spatially coherent components for improved downstream localization and classification.

Experimental Results

Empirical evaluations on VisDrone-DET2019 and AITODv2—both characterized by extreme small object prevalence—establish FSDC-DETR as the current leading method for SOD. On VisDrone-DET2019, FSDC-DETR achieves 31.1 AP (+6.4 over DEIMv2-L) and 21.0 AP$_S$ for small objects (+6.8), while on AITODv2, it reports 32.3 AP (+6.6) and 31.3 AP$_S$ (+6.9), respective gains constituting substantial relative improvement in the regime most sensitive to high-frequency signal preservation.

Further, ablation analyses confirm that DBFSAF, SFS-FF, and FSD-Down offer additive, complementary performance gains, with optimal DBFSAF partial ratio $\gamma=0.5$ yielding the strongest results. Visualizations also demonstrate marked improvement in precision and recall for small targets, with boundaries precisely aligned and background confounds mitigated.

Figure 6: Detection visualization on the VisDrone-DET2019 test split, displaying superior boundary adherence and reduced false positives with FSDC-DETR.

Figure 7: Detection visualization on the AITODv2 test split, highlighting FSDC-DETR's robust tiny object localizations under challenging scenarios.

Practical and Theoretical Implications

FSDC-DETR's demonstration that explicit frequency-spatial disentanglement, adaptive fusion, and frequency-preserving downsampling are critical for high-fidelity SOD challenges the prevailing orthodoxy of solely spatial or implicit spectral processing in vision backbones. This framework suggests that future detection pipelines, especially in domains like remote sensing, autonomous driving, and medical imaging—where small object detection directly influences downstream safety and diagnosis—will trend towards multistage spectral-spatial models with learnable, data-driven frequency alignment at their core.

Theoretically, this work opens new avenues for research into dynamic spectral feature learning, including adaptive kernel design, joint channel-spatial-spectral allocation, and differentiable multi-band attention mechanisms for fine-grained discriminative modeling.

Conclusion

FSDC-DETR presents a frequency-spatial domain collaborative transformer for SOD that reformulates feature representation to explicitly preserve, refine, and propagate high-frequency cues through a principled, unified architecture. By integrating DBFSAF, SFS-FF, and FSD-Down modules, the model achieves leading performance on multiple benchmarks, especially for the challenging small object regime, and demonstrates that frequency-aware modeling is indispensable for precise real-world detection tasks. The methodology and empirical results provide a foundation for future spectral-spatial architectures in detection and dense prediction.