Multispectral Object Detection in Aerial Images

• MODA is a multispectral imaging approach that integrates RGB, infrared, and additional spectral bands for robust aerial object detection in diverse scenarios.
• It utilizes innovative spectral-spatial enhancement and fusion methods—like CSWM and cross-spectral attention—to improve small object detection and suppress background interference.
• Large-scale benchmarks with oriented bounding box annotations validate the efficiency of MODA pipelines in addressing misalignment and computational constraints.

Multispectral Object Detection in Aerial images (MODA) refers to the automated detection and localization of objects in imagery captured by aerial platforms, utilizing multiple spectral bands (commonly RGB and infrared, but also broader multispectral sensors) to enhance robustness under challenging conditions. MODA addresses the limitations of single-spectrum detectors—such as insufficient discrimination in low-illumination or cluttered backgrounds—by exploiting the complementary spectral and spatial cues available in multi-band data. Recent advances encompass new, large-scale benchmarks, sophisticated enhancement and fusion methods, and increasingly efficient model architectures deployed on resource-constrained platforms.

1. MODA Datasets and Benchmark Characteristics

The field was significantly advanced by the release of a dedicated large-scale benchmark, "MODA: The First Challenging Benchmark for Multispectral Object Detection in Aerial Images," comprising 14,041 aerial multispectral images with 330,191 oriented bounding box annotations, covering 8 object classes and 8 spectral bands (395–950 nm). Each image contains, on average, 23.5 annotated objects, with 95% of objects occupying less than 1% of the image area and substantial variation in scale, density, and scenario (urban scenes, varying weather and illumination, extensive occlusion) (Han et al., 10 Dec 2025). The annotation protocol utilizes oriented bounding boxes parameterized as (x_center, y_center, width, height, angle), and split protocols enforce scene-level, non-overlapping train/test division to avoid bleed-through effects.

Additional established aerial datasets include DroneVehicle (RGB-IR, 28,439 image pairs, 5 vehicle classes, 95,3087 bounding boxes), VEDAI (RGB-IR, 1,246 image pairs, 9 object classes), and others such as HOD3K (multispectral, various object types) (Li et al., 9 Sep 2025, Wang et al., 21 Sep 2025, Zuo et al., 9 Nov 2025).

2. Modalities and Sensing Platforms

MODA tasks leverage sensors spanning RGB, thermal infrared (TIR)/LWIR, and extended multispectral cameras typically mounted on UAVs, drones, or stabilized manned aircraft. MODA benchmarks standardize sensor output resolutions (e.g., 1,280×960, re-sized to 1,200×900 for annotation (Han et al., 10 Dec 2025); 640×512 for DroneVehicle (Li et al., 9 Sep 2025)), with ground sample distances ranging down to 4.5 cm/pixel at 100 m altitude. Spectral modalities may range from traditional visible+NIR/IR to 8 bands as in MODA. Sensor co-registration accuracy and spatial alignment are critical to the efficacy of cross-modal fusion algorithms (Zhou et al., 27 Nov 2024, Gallagher et al., 2022).

A summary table of benchmark datasets:

Dataset	Modalities	Images	Classes	Special Features
MODA	8-band MSI	14,041	8	Urban, variable conditions
DroneVehicle	RGB, IR	28,439	5	Oriented bbox, UAV focus
VEDAI	RGB, IR	1,246	9	9 vehicle classes
HOD3K	MSI	3,000+	varies	High-res, multi-scenario

3. Algorithmic and Architectural Innovations

State-of-the-art MODA methods employ elaborate pipelines for multispectral enhancement, feature extraction, and fusion. The core algorithmic frontiers include:

3.1. Spectral-Spatial Enhancement

Low-light and spectral degradation in aerial MODA is mitigated by dual-domain enhancement modules as in DEPF (Li et al., 9 Sep 2025), which integrates:

• Cross-Scale Wavelet Mamba (CSWM): Applies discrete wavelet transform (DWT) to separate low and high-frequency components, performing cross-scale enhancement and global brightness correction on low-freq subbands via linear-time selective state-space modeling (Vision-Mamba SSM).
• Fourier Details Recovery (FDR): Operates in the frequency domain. A spectrum recovery network processes both amplitude and phase after FFT, enabling restoration of fine detail and texture that is often lost in under-exposed RGB regions.

These stages result in enhanced visible images ($I_\mathrm{VE}$), which enter the main feature extraction backbone.

3.2. Feature Fusion Strategies

Fusion architectures are highly diverse, with current trends emphasizing both efficiency and precision. Notable approaches include:

• Priority-Guided Mamba Fusion (PGMF) (Li et al., 9 Sep 2025): Modality features are sorted and serialized by learned priority scores (derived from per-token modality difference), then fused by a linear-time Mamba SSM, ensuring that salient, target-rich tokens are processed first and background redundancy is suppressed.
• Kolmogorov-Arnold Network (KAN) Fusion (Zuo et al., 9 Nov 2025): "Spatial-Frequency Feature Reconstruction" (SFFR) leverages KANs to reconstruct spatial and frequency domain features. The Frequency Component Exchange KAN (FCEKAN) module exchanges cross-modal high-frequency (texture/edge) features, while the Multi-Scale Gaussian KAN (MSGKAN) captures spatial structure and adapts to scale variance (e.g., UAV altitude). The combination of spectral and spatial fusion boosts semantic consistency and detection of small objects.
• Spectral-Spatial Modulation and Cross-Spectral Attention (Han et al., 10 Dec 2025): In OSSDet, a cascade of spectral-spatial joint perception, spectral-similarity aggregation, and cross-spectral attention with object-aware masking refines feature representations. Spectral similarities are exploited to enhance intra-object consistency (SACF), while cross-spectral attention aligns features under explicit object masks, reducing background false positives.

3.3. Lightweight and Deployable Backbones

To address the computational and memory costs of transformer-based fusion, linear-state-space sequence models (e.g., Vision-Mamba SSM) are adopted in DEPF and DMM for $O(N)$ fusion complexity (compared to $O(N^2)$ for self-attention) (Li et al., 9 Sep 2025, Zhou et al., 11 Jul 2024). Large-kernel convolutional backbones with edge-feature guidance (MO R-CNN, (Wang et al., 21 Sep 2025)) efficiently capture spatial context and edge geometry, essential in IR and oblique imagery. Model sizes have been reduced to below 80 MB for real-time UAV deployment at 16 FPS on RTX 3090 (DEPF) (Li et al., 9 Sep 2025).

4. Training Protocols, Augmentation, and Benchmarking

A robust MODA pipeline requires coordinated hyperparameter tuning, dual-modality data augmentation, and rigorous benchmarking (Zhou et al., 27 Nov 2024):

• Training Schedules: Three-stage regime (frozen backbone/fusion branch, gradual unfreezing, then full model training with stepwise LR decay) found optimal. Weight decay, dropout, and pre-training on ImageNet or COCO are standard.
• Alignment and Calibration: Strategies such as LoFTR-based affine feature alignment and SuperFusion dense-pixel warping correct both global and local misregistrations, each contributing 3–6 mAP gain.
• Augmentation: Synchronized geometric transforms, pixel-level (e.g., noise, color/contrast jitter), and low-light or spectral-specific enhancements (e.g., CLAHE, random illumination for TIR) improve generalization by up to 5 mAP, especially for small and low-contrast objects.

Evaluation strictly follows dataset-defined splits; prevalent metrics include [email protected], [email protected]:0.95, oriented bounding box APs, and speed/resource measurements (FPS, params, GFLOPs).

5. Empirical Results and Ablative Analysis

Recent benchmarks demonstrate that advanced fusion and enhancement methods outperform naive or early-fusion baselines:

Method	DroneVehicle [email protected]	VEDAI [email protected]	MODA [email protected]
C²Former	74.2	52.4	N/A
DMM	78.6	70.2	N/A
DEPF	78.9	71.1	N/A
MO R-CNN	78.36	94.2	N/A
SFFR	84.4	N/A	N/A
OSSDet	N/A	N/A	69.0

Comprehensive ablation studies confirm that each major module—cross-scale/frequency enhancement, priority-guided fusion, spectral-similarity or attention, object masking—contributes incrementally to detection accuracy, especially for small objects and challenging conditions.

• In DEPF, DDE (CSWM+FDR) plus PGMF yields +1.2 mAP vs. Z-order fusion (Li et al., 9 Sep 2025).
• In OSSDet, SACF and object-aware masking each provide ~+1% mAP, with cross-spectral attention further boosting tiny object recall (Han et al., 10 Dec 2025).
• KAN-based SFFR demonstrates that nonlinear spatial-frequency fusion is superior to both CNN and Transformer baselines on small-object-heavy datasets (Zuo et al., 9 Nov 2025).

6. Design Challenges and Best Practices

Key technical obstacles specific to MODA include:

• Small Object Detection: High-altitude platforms yield objects occupying few pixels. Multi-scale feature extraction, enhancement of high-frequency detail, and cross-spectral aggregation are crucial (Han et al., 10 Dec 2025, Zuo et al., 9 Nov 2025).
• Background Suppression and Redundancy: Saliency-guided fusion and object-aware masks reduce false positives from complex scenes and overlapping context (Li et al., 9 Sep 2025, Han et al., 10 Dec 2025).
• Spectral Aliasing and Misalignment: Dual-modality calibration, alignment modules, and cross-modal attention mechanisms address geometric mismatch and feature divergence (Zhou et al., 27 Nov 2024, Wang et al., 21 Sep 2025).
• Computational Efficiency: Linear-time fusion models (e.g., Mamba, KAN), model compression, and efficient batch scheduling make real-time operation on embedded UAV platforms feasible (Li et al., 9 Sep 2025, Zhou et al., 11 Jul 2024).

Best practices emerging from empirical studies:

1. Use scene-level splits to prevent data leakage (Han et al., 10 Dec 2025).
2. Choose k=3×3 patches for local spectral aggregation to limit noise.
3. Optimize the activation loss (γ≈0.1) and overall mask loss weight (α≈0.6) in object-aware modules.
4. Favor feature-level fusion over pixel- or decision-level, with synchronized augmentation across domains (Zhou et al., 27 Nov 2024).
5. Prioritize explicit mask and cross-spectral attention modules for tiny-object scenarios.

7. Future Directions

Open research problems and promising directions in MODA include:

• Spectral Augmentation: Automated band dropout, spectral mixup, or advanced photometric transforms for robust training (Han et al., 10 Dec 2025).
• Transformer-based Cross-Spectral Methods: Exploration of transformer backbones with efficient, cross-spectral self- or cross-attention for finer domain interaction.
• Generalization Across Sensors and Domains: Benchmarking transfer accuracy across sensor types, differing flight altitudes, weather, and scene distributions to foster deployable, adaptive MODA systems.
• Hardware Acceleration: Implementation of FFT, DWT, and SSM modules on custom UAV hardware to overcome inference bottlenecks (Li et al., 9 Sep 2025).
• Integration of Additional Modalities: Extending MODA frameworks to include SAR, LiDAR, and other orthogonal sensor data, with possible end-to-end backbone–enhancement codesign (Li et al., 9 Sep 2025).

The introduction of the MODA benchmark and frameworks such as OSSDet, combined with dual-domain enhancement, spectral-spatial cascades, object-aware guidance, and computationally efficient fusion, establishes a clear standard for future aerial multispectral object detection research (Han et al., 10 Dec 2025, Li et al., 9 Sep 2025, Zuo et al., 9 Nov 2025, Zhou et al., 27 Nov 2024, Wang et al., 21 Sep 2025).