Priority-Guided Mamba Fusion (PGMF)

• The paper introduces PGMF, which converts cross-modal discrepancies into a token-ranking signal using a learned Priority Score Network.
• PGMF serializes multispectral features by priority to emphasize local targets and reduce background redundancy compared to traditional Z-order methods.
• Empirical results on UAV datasets show improved mAP metrics with PGMF, validating its efficiency and adaptability in low-light multispectral object detection.

Priority-Guided Mamba Fusion Module (PGMF) is the fusion component in DEPF that replaces standard order-agnostic or global-first sequence construction with a target-prioritized token scanning strategy for multispectral RGB–infrared feature fusion. It was introduced for UAV multispectral object detection under low-light conditions, where low-light RGB features differ substantially from infrared features and conventional fusion or serialization schemes such as Z-order scanning tend to emphasize global context while suppressing local dependencies. Within DEPF, PGMF is the feature-alignment and multimodal fusion stage that follows the Dual-Domain Enhancement Module (DDE), and its stated role is to make fusion more effective in low-light UAV scenes by improving target saliency before sequence modeling (Li et al., 9 Sep 2025).

1. Problem setting and motivation

DEPF identifies three challenges in multispectral remote sensing object detection for UAVs: low-light remote sensing images reduce the complementarity during multi-modality fusion; local small target modeling is interfered with redundant information in the fusion stage easily; and transformer-based methods are hard to apply on the UAV platform because of quadratic computational complexity (Li et al., 9 Sep 2025).

PGMF is introduced to address the second and third of these limitations in the fusion stage. The motivating failure mode is that, for UAV remote sensing, objects are often small and sparsely distributed, yet conventional fusion and serialization schemes such as Z-order scanning tend to emphasize global context while suppressing local dependencies. This causes background redundancy to dominate and weakens small-target modeling. PGMF therefore aims to make the sequence start from tokens more likely to belong to local targets, so that the Mamba state-space model can aggregate useful target evidence first and then absorb background information later.

This design is explicitly tied to low-light multispectral detection. The modality discrepancy between RGB and infrared is treated not merely as a nuisance but as a source of localization cues. At the same time, the formulation acknowledges that modality differences can also reflect regions that are being attended to incorrectly. PGMF therefore does not directly equate discrepancy with targetness; instead, it uses a learned Priority Score Network to map discrepancy into token importance for fusion.

2. Position within DEPF

DEPF first applies the Dual-Domain Enhancement Module to low-light RGB images and then feeds enhanced RGB and infrared features into PGMF for fusion (Li et al., 9 Sep 2025). DDE contains Cross-Scale Wavelet Mamba and the Fourier Details Recovery block. CSWM applies cross-scale mamba scanning for the low-frequency components to enhance the global brightness of images, while FDR constructs spectrum recovery network to enhance the frequency spectra features for recovering the texture-details. PGMF follows this enhancement stage rather than operating as a standalone fusion head.

In the overall detector, the architecture uses a Vision-Mamba backbone and an $\text{S}^2\text{ANet}$ head, with training on $512\times640$ DroneVehicle inputs and $512\times512$ VEDAI inputs. The loss is given as

$$\mathcal{L}=\alpha \mathcal{L}_{det}+\beta \mathcal{L}_{reg},\quad \alpha=\beta=1.0.$$

The detector is reported at 80.3 MB parameters, 108.3 GFLOPs, and 16 FPS on an RTX 3090 for $512\times640$ input, and the inference speed is explicitly noted to be affected more by the DDE components, namely 2D-DWT and FFT, than by the fusion mechanism itself (Li et al., 9 Sep 2025).

A concise summary of module placement is given below.

Stage	Component	Stated role
Enhancement	DDE	Enhances low-light RGB images
Fusion	PGMF	Feature-alignment and multimodal fusion
Detection	Vision-Mamba + $\text{S}^2\text{ANet}$	Object detection pipeline

This placement suggests that PGMF is intended to exploit improved RGB saliency generated upstream, rather than to compensate alone for degraded visual input.

3. Priority scanning formulation

The core idea of PGMF is priority scanning. Instead of flattening features in a fixed spatial order, PGMF computes a per-token priority from the modality difference and uses that priority to sort tokens before serialization (Li et al., 9 Sep 2025).

The latent features from the RGB and infrared inputs are defined as $\mathcal{F}_{vl}$ and $\mathcal{F}_{il}$. Their difference is

$$\varDelta \mathcal{F}=\mathcal{F}_{vl}-\mathcal{F}_{il}.$$

This difference is then fed into a Priority Score Network:

$PMat=PSN\left( \varDelta \mathcal{F} \right).$

The formulation explicitly notes that $512\times640$0 contains both target location cues and regions that are being attended to incorrectly, so the PSN learns a score map that reflects token importance for fusion.

The priority-enhanced latent features are constructed by residual addition:

$512\times640$1

These scores are then used to sort tokens in descending priority so that the sequence begins with target-bearing tokens and ends with background tokens. Architecturally, the RGB feature map $512\times640$2 and infrared feature map $512\times640$3 are first mapped to a latent space using small MLPs, after which the modality difference is computed and the priority scores are predicted.

The significance of this formulation lies in the conversion of cross-modal discrepancy into a token-ranking signal. A plausible implication is that PGMF treats token order not as a fixed implementation detail but as a learnable fusion variable.

4. Architectural mechanism

The Priority Score Network is described as consisting of three DWCONV-SiLU layers, one LayerNorm layer, and one Linear layer (Li et al., 9 Sep 2025). After scoring, each modality is serialized using the learned priority order:

$512\times640$4

The two serialized streams are fused as

$512\times640$5

where $512\times640$6 denotes the reverse sequence of the infrared stream.

The reverse ordering of the infrared sequence is not incidental. The formulation states that the two modalities may be scanned in opposite directions depending on their sorted priority order, and that this is necessary because the modality-specific priority rankings are not always consistent. The fused sequence is then processed through the state-space model:

$512\times640$7

The final fused output is obtained by residual aggregation with the original modality features:

$512\times640$8

This mechanism differentiates PGMF from simple concatenation or addition-based fusion because it uses score-driven serialization rather than a fixed spatial order. It also differs from transformer-based fusion such as $512\times640$9 by avoiding quadratic attention cost and instead using a linear-complexity Mamba/SSM backbone for the fusion sequence. Relative to common Mamba vision models that use generic Z-order scanning, PGMF explicitly biases the scan toward local targets, addressing the small-object and background-interference problem that Z-order can exacerbate (Li et al., 9 Sep 2025).

5. Empirical evidence and visual interpretation

The direct ablation evidence on DroneVehicle isolates the contribution of PGMF. Replacing Z-order fusion with PGMF improves performance from 77.8/49.7 to 78.1/50.8 for mAP@0.5 / [email protected]:0.95 in the simplest comparison. In later full-model comparisons, adding PGMF on top of the enhancement modules leads to the best result, reaching 78.9 [email protected] and 51.2 [email protected]:0.95 (Li et al., 9 Sep 2025).

The reported interpretation is that priority scanning is more effective than Z-order scanning and that fusion improves further when the input RGB image has been enhanced by DDE. The accompanying visualization is described as showing that Z-order fusion tends to focus on background, whereas priority-guided fusion shifts attention toward the actual objects.

These results are specific to the DEPF setting on DroneVehicle and should be read in that context. The article does not report exact tensor shape changes inside PGMF, and it does not separate the runtime of PGMF from the rest of DEPF. However, the reported system-level profile and the ablation outcomes support the stated claim that the fusion mechanism benefits small-target multispectral UAV detection while remaining compatible with a linear-complexity sequence model.

6. Relation to adjacent Mamba fusion research

PGMF is presented as an attempt to improve Mamba for multimodal fusion in low-light UAV multispectral detection (Li et al., 9 Sep 2025). A related but distinct development appears in GMF-Drive, where the corresponding fusion architecture is called GM-Fusion, with the Mamba/SSM core referred to more specifically as BEV-SSM or AwareSSM (Wang et al., 8 Aug 2025).

The two modules address different domains and input modalities. PGMF fuses RGB and infrared features for UAV multispectral object detection, whereas GM-Fusion operates in end-to-end autonomous driving with synchronized camera images and LiDAR point clouds. PGMF prioritizes tokens using modality difference and a learned score map, then serializes them in descending priority order. By contrast, GM-Fusion expresses its “priority-guided” behavior through directional weighting, distance decay, adaptive fusion over scanning patterns, and BEV-oriented spatial priors rather than through a separately named priority-scoring block.

The comparison is useful because both systems position state-space models as alternatives to transformer fusion. In GMF-Drive, the SSM-based BEV-SSM is claimed to have linear $512\times512$0 complexity and to be more compatible with BEV structure than transformer self-attention, while ablations show that task-specific SSMs can surpass a general-purpose transformer in both performance and efficiency (Wang et al., 8 Aug 2025). This suggests a broader pattern in which Mamba-like fusion modules are being specialized by domain-specific token ordering or spatial priors rather than applied as generic sequence processors.

7. Interpretation, novelty, and scope

PGMF’s novelty is identified as converting cross-modal discrepancy into a token-ranking signal, using that ranking to control serialization, and then applying linear-complexity SSM fusion to preserve target details while remaining lighter than transformer-based alternatives (Li et al., 9 Sep 2025). In summary form, it is the modality-difference-driven, priority-sorted Mamba fusion block that lets DEPF process RGB and infrared features in an order that privileges object tokens over background tokens.

Several misconceptions can be clarified from the formulation. PGMF is not a standalone fusion head detached from the rest of DEPF; it operates after DDE and is part of a larger detector pipeline. It is not described as replacing all multimodal processing in the detector; rather, it is the fusion stage between enhanced RGB and infrared features. It also does not infer token importance directly from either modality alone, since the priority matrix is derived from the difference $512\times512$1 and learned by PSN.

The scope of the reported evidence is also specific. The ablation results establish effectiveness on DroneVehicle, and DEPF is also evaluated on VEDAI, where the overall detector is reported to perform well compared with state-of-the-art methods (Li et al., 9 Sep 2025). Beyond those claims, any generalization to other multispectral fusion problems should be treated as an inference. What the available evidence directly supports is a precise methodological contribution: a learned, modality-difference-driven serialization rule for Mamba-based RGB–infrared fusion in low-light UAV remote sensing.