D$^3$R-DETR: Dual-Domain Tiny Object Detector

• The paper introduces a dual-domain density refinement strategy that integrates spatial and frequency cues to guide transformer attention and improve localization.
• It leverages a density-aware pipeline with Masked Window Attention Sparsification and Progressive Adaptive Query Initialization to focus computation on high-density regions.
• The architecture uses an HGNetv2-B0 backbone with Fractional Gabor Kernels to enhance feature extraction, yielding robust convergence and superior AP on tiny object benchmarks.

D$^3$R-DETR is a detection transformer (DETR) variant designed for high-precision tiny object detection, particularly in the context of low-resolution and high-density aerial imagery. It introduces a dual-domain density refinement strategy that leverages both spatial and frequency cues to enhance the density map prediction, which is then used to guide the transformer’s attention mechanisms and query initialization. This framework achieves superior performance to previous state-of-the-art detectors on challenging tiny object benchmarks by focusing the model’s representational and computational resources on crowded, information-rich areas and refining localization through multilayer fusion.

1. Architectural Foundations

D$^3$R-DETR is built atop Dome-DETR, utilizing an HGNetv2-B0 CNN backbone to extract multi-scale feature maps ($F_1,\ldots,F_4$), a single-layer transformer encoder, and a deformable transformer decoder. Two primary innovations distinguish D$^3$R-DETR from its predecessors:

• Dual-Domain Fusion Module (D$^2$FM): This module employs two parallel branches for low-level feature enhancement—a Frequency Processing Unit (FPU) using Fractional Gabor Kernels (FrGK) for selective frequency analysis, and a Dilated Spatial Processing Unit (DilatedSPU) for multi-scale spatial context aggregation via dilated convolutions and channel attention. The outputs from both branches are fused with a point-wise convolution.
• Density-Aware Pipeline: The density head, consuming D$^2$FM’s fused output, produces a high-resolution density map $\hat{D} \in \mathbb{R}^{H\times W}$. This map orchestrates two components:
  • Masked Window Attention Sparsification (MWAS): Applied in the encoder, MWAS masks out low-density feature windows, enforcing computation on likely object-rich regions.
  • Progressive Adaptive Query Initialization (PAQI): Used by the decoder for density-guided initialization and selection of object queries, focusing modeling power on high-density areas.

These innovations couple global and local context, resulting in precise localization and robust convergence under severe crowding and visual ambiguity (Wen et al., 6 Jan 2026).

2. Detailed Module Descriptions and Mathematical Formulations

2.1 Dual-Domain Fusion Module (D$^2$FM)

D$^2$FM enhances the lowest-level feature map through:

• FPU: Generates $F_{\text{FPU}}$ by stacking $N$ FrGK filters with different scales and orientations, each formulated as

$$G(x, y) = \exp\left(-\frac{x'^2 + y'^2}{2\sigma^2}\right)\cos(2\pi f\, x')$$

where $x' = x\cos\theta + y\sin\theta$, $y' = -x\sin\theta + y\cos\theta$. Multi-angle, multi-scale convolution (N=12, optimal empirically) are applied, and their outputs are fused by a pointwise convolution.

• DilatedSPU: Performs channel splitting (two groups), each processed by 3x3 convolutions with dilations 1 and 2, then summed 'residually,' followed by channel attention and a final 1x1 convolution to obtain $F_{\text{DS}}$.
• Fusion: The two branches are concatenated and fused by pointwise convolution:

$$F_{\text{out}} = \text{PWC}([F_{\text{FPU}}; F_{\text{DS}}])$$

2.2 Density Head

$F_{\text{out}}$ is mapped to a scalar density map by a lightweight head (3–5 convolutional/upsampling layers). The predicted density map supervises both encoder masking and decoder query positioning.

2.3 Loss Function

The optimized loss is

$$\mathcal{L} = \mathcal{L}_{\mathrm{cls}} + \lambda_{\text{bbox}}\mathcal{L}_{\text{bbox}} + \lambda_{\text{den}}\mathcal{L}_{\text{den}}$$

where $\mathcal{L}_{\mathrm{cls}}$ is cross-entropy/focal loss across $K$ queries and $C$ classes, $\mathcal{L}_{\text{bbox}}$ combines $L_1$ and Generalized IoU losses for positive queries, and $\mathcal{L}_{\text{den}}$ is the Density Recall Focal Loss (DRFL) as introduced in Dome-DETR. Typical weights: $\lambda_{\text{bbox}}=5$, $\lambda_{\text{den}}=1$.

3. Training Setup and Evaluation Protocols

3.1 Data and Preprocessing

The system is primarily evaluated on the AI-TOD-v2 remote sensing dataset (8 classes, $>750,000$ instances, mean object size $12.7\pm5.6$ pixels). Multi-scale features and advanced augmentation are central to ensuring performance and generalizability.

3.2 Hyperparameters

• Hardware: 4×NVIDIA RTX 4090 GPUs
• Batch size: 16 (4 per GPU)
• Optimization: AdamW, $lr=1\times10^{-4}$, $weight\_decay=1\times10^{-4}$
• Epoch schedule: 120 main, 25 fine-tuning; learning rate decayed at epoch 100
• Initialization: HGNetv2-B0 is ImageNet pretrained; transformer layers are randomly initialized
• Stabilization: The first few backbone stages are frozen initially; gradient clipping (norm <0.1) prevents density head instabilities

This approach enables stable optimization even with the added complexity of D$^2$FM and the density pipeline.

4. Experimental Results and Ablations

4.1 Comparison with State-of-the-Art

On AI-TOD-v2, D$^3$R-DETR achieves state-of-the-art performance among tiny object detectors, notably improving average precision (AP) over both CNN and transformer-based methods:

Method	Backbone	AP	AP$_{50}$	AP$_{75}$	AP$_{vt}$	AP$_{t}$	AP$_{s}$	AP$_{m}$
Dome-DETR*	HGNetv2-B0	28.7	62.0	22.8	14.6	28.1	34.2	42.2
D$^3$R-DETR (Ours)	HGNetv2-B0	31.3	65.1	26.2	16.6	30.8	36.8	44.7

Absolute gains (vs Dome-DETR*): +2.6 AP (overall), +3.1 AP$_{50}$, +3.4 AP$_{75}$ (Wen et al., 6 Jan 2026).

4.2 Ablation on Fractional Kernels

Fractional Filter Type	AP	AP$_{50}$	AP$_{75}$
Baseline (no D$^2$FM)	28.7	62.0	22.8
Haar	30.0	63.4	24.2
Fourier	30.3	63.8	24.7
Gabor	31.3	65.1	26.2

Fractional Gabor kernels offer the highest performance, confirming the efficacy of orientation- and scale-aware local frequency processing.

4.3 Training Dynamics

The DRFL converges ~20% faster and more stably relative to Dome-DETR’s baseline density extractor, resulting in accelerated AP improvements.

5. Implementation Notes and Best Practices

• Integrating D$^2$FM increases overhead by <5% but yields substantial improvements in density quality and detection performance.
• Four orientations and three scales for FrGK ($N=12$) are empirically optimal.
• Freezing early backbone layers initially is advantageous for stabilization.
• Applying MWAS and PAQI with hyperparameters inherited from Dome-DETR maximizes end-to-end synergy.
• Gradient clipping at norm <0.1 prevents density-induced training instability.

A plausible implication is that spatial-frequency fusion in the density estimator is a key driver for robust convergence under severe object crowding.

6. Context, Impact, and Significance

D$^3$R-DETR advances the state of the art in tiny object detection for remote sensing by integrating spatial and frequency domain information, yielding a uniquely precise density map leveraged throughout the architecture. This density-awareness enables both targeted attention allocation in transformers and more accurate query initialization. The quantitative improvements are matched by faster and more stable training dynamics relative to prior art. D$^3$R-DETR’s architectural principles—especially dual-domain fusion for density cues—represent an effective strategy for adapting transformer-based detectors to extreme scales and densities found in aerial image understanding scenarios (Wen et al., 6 Jan 2026).