WaveMamba: Wavelet Fusion for RGB-IR Detection

• WaveMamba is a frequency-domain fusion approach that decomposes RGB and IR features using one-level DWT to separate low- and high-frequency information.
• It employs a dual-stage WaveMamba Fusion Block (WMFB) with LMFB and high-frequency enhancement to integrate complementary modality cues.
• The method reconstructs high-resolution detection features via IDWT, yielding mAP gains up to 4.5% across multiple RGB-IR detection benchmarks.

Searching arXiv for the specified paper and closely related wavelet–Mamba work for citation grounding. arXiv search query: (Zhu et al., 24 Jul 2025) WaveMamba Wavelet Mamba RGB Infrared Object Detection WaveMamba is a cross-modality fusion method for RGB-infrared object detection that decomposes visible and infrared features with a one-level Discrete Wavelet Transform (DWT), performs frequency-specific fusion, and reconstructs a full-resolution detection feature with Inverse DWT (IDWT) inside an improved YOLOv8-style detection head (Zhu et al., 24 Jul 2025). Its central unit is the WaveMamba Fusion Block (WMFB), which separates low-frequency and high-frequency processing: low-frequency components are fused by a Mamba-based Low-frequency Mamba Fusion Block (LMFB), while high-frequency components are combined by an “absolute maximum” rule. Reported experiments describe average mAP improvements of 4.5% on four benchmarks and consistent gains across multiple RGB-IR detection settings (Zhu et al., 24 Jul 2025).

1. Task setting and design premise

WaveMamba is designed for RGB-IR object detection, where visible and infrared imagery provide complementary characteristics. The method is built on a two-stream backbone, such as ResNet50, YOLOv5, or YOLOv8, that processes RGB and IR inputs in parallel. Its design premise is that fusion should not be performed on undifferentiated feature tensors; instead, low-frequency and high-frequency components should be treated as distinct carriers of information.

The paper characterizes this in explicitly frequency-aware terms. Low-frequency content is associated with structure, while high-frequency content is associated with local detail. The stated strength is that the framework exploits complementary frequency priors: IR for structure in the low-frequency regime and RGB for detail in the high-frequency regime (Zhu et al., 24 Jul 2025). This motivates a decomposition-first architecture rather than a purely spatial or channel-wise fusion scheme.

A plausible implication is that WaveMamba is best understood not as a generic Mamba backbone for detection, but as a frequency-domain fusion strategy layered onto standard two-stream detectors. This is consistent with the reported use of existing backbones and with the statement that the rest of the YOLOv8 head remains unchanged after the IDWT-based reconstruction step.

2. Architectural organization

WaveMamba begins with a two-stream backbone that extracts modality-specific features from RGB and IR inputs. After the first two convolutional stages, each modality’s feature map is passed through a one-level DWT, yielding low-frequency and high-frequency sub-bands. In the notation of the paper,

$$(F^{2}_{L,\mathrm{RGB}},F^{2}_{H,\mathrm{RGB}})=\mathrm{DWT}(f^2_{\mathrm{RGB}}), \qquad (F^{2}_{L,\mathrm{IR}},F^{2}_{H,\mathrm{IR}})=\mathrm{DWT}(f^2_{\mathrm{IR}}).$$

Fusion is inserted after stages 2, 3, and 5, corresponding to features at $P_2,P_3,P_5$. The paper reports that this combination is optimal: earlier layers lack sufficient semantics, whereas later layers become too coarse (Zhu et al., 24 Jul 2025). Three WMFBs are used, and an ablation further states that 3 blocks $>$ 2 $>$ 1.

The final fused sub-bands are fed to an improved YOLOv8-style detection head. Rather than reconstructing resolution by naïve up-sampling, the method uses IDWT to recover a single high-resolution feature map from fused low-frequency and high-frequency components. Bounding-box prediction and classification/regression heads are then applied in the conventional way.

This architecture distributes computation according to frequency role. Low-frequency maps are subjected to deeper learned fusion, while high-frequency bands are handled by a parameter-free selection rule. This suggests a deliberate asymmetry: the model spends representational capacity where global structure and cross-modal consistency matter most, and uses an inexpensive mechanism where edge/detail retention is the priority.

3. Wavelet decomposition and reconstruction

WaveMamba adopts the simplest Haar wavelet. In one dimension, the low-pass and high-pass filters are

$$L=\tfrac{1}{\sqrt{2}}[1,1], \qquad H=\tfrac{1}{\sqrt{2}}[1,-1].$$

For a 2D feature map $I$, a one-level forward transform produces one low-frequency sub-band and three high-frequency sub-bands: $$(F_L,F_H)=\mathrm{DWT}(I), \qquad F_L=\{F_{LL}\}, \quad F_H=\{F_{LH},F_{HL},F_{HH}\}.$$

The paper describes the forward transform via depth-wise convolution with low-pass and high-pass filters followed by spatial down-sampling by 2. The inverse transform recombines sub-bands through up-sampling and convolution with the transposed filters. In the detection pipeline, this reconstruction is not merely a mathematical inverse; it is incorporated as an architectural mechanism to preserve multiscale wavelet details in the final feature map (Zhu et al., 24 Jul 2025).

The default basis is Haar, but other wavelet bases were tested. Sym3 and DB3 are reported to remain within $\pm 0.3\%$ mAP of Haar, and Haar is identified as the best choice in the ablation (Zhu et al., 24 Jul 2025). This suggests that the gain derives more from the frequency-separation strategy itself than from a highly specialized wavelet basis.

4. WaveMamba Fusion Block

The WMFB has two parallel streams: LMFB for low-frequency fusion and High-Frequency Enhancement (HFE) for high-frequency fusion.

Low-frequency Mamba Fusion Block

LMFB receives two low-frequency maps, $F^i_{L,\mathrm{RGB}}$ and $F^i_{L,\mathrm{IR}}$, and performs fusion in two stages.

The first stage is Shallow Fusion Mamba (SFM). It applies a channel-swapping operation $P_2,P_3,P_5$0 that exchanges half of the channels across modalities: $P_2,P_3,P_5$1 Each swapped tensor is then passed through two stacked VSS blocks, described as 2D Selective Scan plus state-space S6, producing

$P_2,P_3,P_5$2

The second stage is Deep Fusion Mamba (DFM). It alternately treats one modality as primary and the other as auxiliary. The primary stream is split into two flows: one passes through $P_2,P_3,P_5$3 depth-wise convolution $P_2,P_3,P_5$4 SiLU $P_2,P_3,P_5$5 SS2D $P_2,P_3,P_5$6 LayerNorm, while the other passes through SiLU only. A gated attention mechanism uses the second flow of the primary stream to gate both the first flow and the auxiliary features, then sums them. Symbolically,

$P_2,P_3,P_5$7

The paper attributes low-frequency fusion quality to both stages. On M³FD with a YOLOv8 backbone, removing SFM causes a $P_2,P_3,P_5$8 mAP$P_2,P_3,P_5$9 drop, and removing DFM causes a $>$0 mAP$>$1 drop (Zhu et al., 24 Jul 2025). Low-frequency average fusion is also reported to underperform LMFB by $>$2 mAP$>$3.

High-Frequency Enhancement

High-frequency sub-bands $>$4 are fused by selecting, at each pixel, the coefficient with the larger magnitude, described as an “absolute maximum” strategy: $>$5 Here $>$6 is a binary map equal to 1 where $>$7 and 0 otherwise, and $>$8 denotes element-wise multiplication.

This fusion is parameter-free, and the paper states that it gates out noise without extra cost (Zhu et al., 24 Jul 2025). In ablation, replacing HFE with average high-frequency fusion causes a $>$9 mAP$>$0 drop.

5. IDWT-based detection head and implementation profile

After the last WMFB, the detector has fused low-frequency maps $>$1 and a fused high-frequency map $>$2. The two low-frequency maps are first summed into a single $>$3, and IDWT is then applied to $>$4 to recover a high-resolution feature map. The remainder of the YOLOv8 head—anchor prediction and classification/regression heads—remains unchanged (Zhu et al., 24 Jul 2025).

The paper argues that this IDWT step reduces information loss relative to simple up-sampling, and reports a $>$5 mAP$>$6/mAP improvement for the improved head over the original head in ablation. A plausible interpretation is that the detection head is not a separate innovation detached from fusion; it is the reconstruction counterpart that makes the frequency decomposition operationally useful at the output side.

Implementation details are specified for both training and inference. Training uses a single A800 GPU, batch size 16, $>$7 input, 250 epochs, and SGD with $>$8, momentum $>$9, weight_decay $$L=\tfrac{1}{\sqrt{2}}[1,1], \qquad H=\tfrac{1}{\sqrt{2}}[1,-1].$$0, together with default YOLOv8 augmentations. Inference is measured with batch size 32 on an A800 GPU, with 40 ms per image pair for the YOLOv8 version (Zhu et al., 24 Jul 2025).

6. Experimental performance, ablations, and qualitative behavior

WaveMamba is evaluated on M³FD, DroneVehicle, LLVIP, and FLIR-Aligned, with VEDAI and KAIST reported in the supplementary. The principal metrics are mAP$$L=\tfrac{1}{\sqrt{2}}[1,1], \qquad H=\tfrac{1}{\sqrt{2}}[1,-1].$$1 and mAP$$L=\tfrac{1}{\sqrt{2}}[1,1], \qquad H=\tfrac{1}{\sqrt{2}}[1,-1].$$2.

Dataset	Result	Note
M³FD	92.1% mAP$$L=\tfrac{1}{\sqrt{2}}[1,1], \qquad H=\tfrac{1}{\sqrt{2}}[1,-1].$$3 / 64.4% mAP	+5.5% / +5.1% over prior best
DroneVehicle	79.8% mAP$$L=\tfrac{1}{\sqrt{2}}[1,1], \qquad H=\tfrac{1}{\sqrt{2}}[1,-1].$$4 / 60.5% mAP	1st among all methods
LLVIP	98.3% mAP$$L=\tfrac{1}{\sqrt{2}}[1,1], \qquad H=\tfrac{1}{\sqrt{2}}[1,-1].$$5 / 66.0% mAP	pedestrian low-light
FLIR-Aligned	88.4% mAP$$L=\tfrac{1}{\sqrt{2}}[1,1], \qquad H=\tfrac{1}{\sqrt{2}}[1,-1].$$6 / 48.1% mAP	precision 84.2%, recall 80.9%, F1 82.5%

On FLIR-Aligned, the method is additionally reported to outperform CrossFormer by $$L=\tfrac{1}{\sqrt{2}}[1,1], \qquad H=\tfrac{1}{\sqrt{2}}[1,-1].$$7 points F1 while cutting 294 M parameters. On VEDAI and KAIST, the paper reports top-3 or better performance with minimal overhead (Zhu et al., 24 Jul 2025).

The ablation study on M³FD with a YOLOv8 backbone isolates the main components. The improved head contributes $$L=\tfrac{1}{\sqrt{2}}[1,1], \qquad H=\tfrac{1}{\sqrt{2}}[1,-1].$$8 mAP$$L=\tfrac{1}{\sqrt{2}}[1,1], \qquad H=\tfrac{1}{\sqrt{2}}[1,-1].$$9/mAP over the original head. Removing SFM or DFM degrades performance, and replacing LMFB or HFE with average fusion also reduces mAP$I$0. The best insertion positions are $I$1, three WMFBs outperform fewer blocks, and different wavelet bases remain close, with Haar best (Zhu et al., 24 Jul 2025).

Qualitative analyses are aligned with these quantitative results. Grad-CAM heatmaps on M³FD, DroneVehicle, LLVIP, and FLIR show that WaveMamba focuses tightly on object regions and suppresses distracting background. Visual detections in adverse weather, low light, and dense scenes show fewer false positives and false negatives than competing fusion and transformer methods. Wavelet-domain visualizations indicate that fused low-frequency maps carry sharper silhouettes and that high-frequency maps retain richer edges and detail, which the paper associates with improved recall for small and occluded objects (Zhu et al., 24 Jul 2025).

7. Limitations, future work, and relation to similarly named models

The paper identifies several limitations. The wavelet transform adds minor computational overhead and requires careful alignment, specifically strict pixel pairing between RGB and IR inputs. Haar is simple, but more advanced wavelet bases may yield marginally better detail and require empirical tuning. The framework assumes well-registered RGB/IR pairs, so misalignment or parallax may degrade performance (Zhu et al., 24 Jul 2025).

Future directions are explicitly outlined. These include learnable wavelet bases or steerable multiscale transforms, extension to unaligned or event-based multimodal data such as fusing LiDAR ranges in the wavelet domain, end-to-end optimization of fusion blocks and detection head in self-supervised or domain-adaptive settings to handle cross-sensor shifts, and real-time lightweight variants through WMFB pruning or dynamic block selection at inference time (Zhu et al., 24 Jul 2025).

A common source of confusion is nomenclature rather than method. The name “WaveMamba” is not unique in recent arXiv literature. Closely related names designate distinct architectures for UHD low-light image enhancement (Zou et al., 2024), hyperspectral image classification (Ahmad et al., 2024), image debanding (Wang et al., 15 Aug 2025), image dehazing under the name WDMamba (Sun et al., 7 May 2025), and sea surface temperature super-resolution under the name Wavelet-Assisted Mamba (Chen et al., 29 Sep 2025). This suggests that, in technical discussion, the RGB-IR detector is most precisely identified by its full title, “WaveMamba: Wavelet-Driven Mamba Fusion for RGB-Infrared Object Detection” (Zhu et al., 24 Jul 2025).