MPFR Module for Visible-Infrared Re-ID

• The paper introduces MPFR, which refines multi-scale features using spatially-aware masks to highlight modality-specific identity cues.
• MPFR extracts, aligns, and fuses features from multiple backbone stages using convolutions and attention-like operations for robust identity representation.
• Empirical tests on SYSU-MM01 show notable gains, with Rank-1 improvements up to 77.51% and mAP gains reaching 74.16%.

The Multi-Perception Feature Refinement (MPFR) module is a neural network component introduced to enhance Visible-Infrared Person Re-Identification (VI-ReID) by explicitly mining and aggregating modality-specific identity cues from the shallower layers of a shared feature extractor. Unlike approaches that focus exclusively on modality-invariant embeddings, MPFR targets the preservation and fusion of multi-scale discriminative features that are often suppressed or neglected in standard architectures. Its design emphasizes the refinement and spatially-aware selection of features from multiple perceptive levels, making them available for subsequent distillation and enhancement by downstream modules (Zhang et al., 4 Dec 2025).

1. Architectural Position and High-Level Role

MPFR is positioned downstream of the shared backbone (ResNet-50, stages 2–4) and immediately precedes the Semantic Distillation Cascade Enhancement (SDCE) module in the Identity Clue Refinement and Enhancement (ICRE) network. Its core functions are threefold: to extract features from three successive backbone stages at different resolutions, align all to a common scale and channel dimension, learn spatial "importance" masks highlighting the most identity-informative regions at each perceptual scale, and fuse these into a single identity-guided feature map. The approach is fully branch-free, relying on linear, convolutional, and attention-like operations.

2. Feature Extraction, Alignment, and Fusion Process

The MPFR module operates on three specific feature maps from the shared backbone:

• $f_\ell \in \mathbb{R}^{B, C_2, H_2, W_2}$ from stage 2 (e.g., $C_2=512$, $H_2=48$, $W_2=24$),
• $f_m \in \mathbb{R}^{B, C_3, H_3, W_3}$ from stage 3 ($C_3=1024$, $H_3=24$, $W_3=12$),
• $f_h \in \mathbb{R}^{B, C_4, H_4, W_4}$ from stage 4 ($C_4=2048$, $C_2=512$0, $C_2=512$1).

2.1 Feature-Scale Alignment

Each feature is processed by a dedicated $C_2=512$2 ConvBlock (convolution, batch normalization, ReLU) to yield aligned features $C_2=512$3, all reshaped to size $C_2=512$4 by setting stride $C_2=512$5 for $C_2=512$6, $C_2=512$7 for $C_2=512$8, and $C_2=512$9 for $H_2=48$0.

2.2 Spatial-Mask Generation

For each $H_2=48$1:

• Channel-pooling generates two spatial maps per scale: $H_2=48$2 and $H_2=48$3, concatenated across the channel dimension.
• A triple-branch of $H_2=48$4 convolutions with dilation rates $H_2=48$5, $H_2=48$6, and $H_2=48$7 is applied to these pooled maps, and the outputs are summed to produce spatial attention logits $H_2=48$8.

2.3 Softmax Spatial Weighting

At each spatial position $H_2=48$9, a softmax is computed across the three scales, resulting in normalized spatial weights $W_2=24$0 such that $W_2=24$1 for each position.

2.4 Weighted Fusion and Channel Restoration

A weighted sum $W_2=24$2 fuses the features across scales, which is then passed through a $W_2=24$3 ConvBlock (to $W_2=24$4 channels) to restore depth before output.

The data flow can be summarized as:

Stage	Operation	Output Size
Backbone features	$W_2=24$5, $W_2=24$6, $W_2=24$7	512, 1024, 2048 ch
Alignment	3×3 ConvBlock, stride 4,2,1	[256,12,6] each
Masking	Channel pooling + 3 dilated convs	[1,12,6] per scale
Weighting	Softmax over scales at each position	[1,12,6] per scale
Fusion	Weighted sum $W_2=24$8 final ConvBlock	[2048,12,6]

3. Mathematical Formulation

Formally, MPFR can be expressed as follows:

• Alignment: $W_2=24$9, with $f_m \in \mathbb{R}^{B, C_3, H_3, W_3}$0 as $f_m \in \mathbb{R}^{B, C_3, H_3, W_3}$1 convolutional weights, $f_m \in \mathbb{R}^{B, C_3, H_3, W_3}$2 as stride.
• Mask Generation: For each $f_m \in \mathbb{R}^{B, C_3, H_3, W_3}$3:
  • $f_m \in \mathbb{R}^{B, C_3, H_3, W_3}$4, $f_m \in \mathbb{R}^{B, C_3, H_3, W_3}$5
  • $f_m \in \mathbb{R}^{B, C_3, H_3, W_3}$6
  • $f_m \in \mathbb{R}^{B, C_3, H_3, W_3}$7
• Softmax Weighting: $f_m \in \mathbb{R}^{B, C_3, H_3, W_3}$8, for $f_m \in \mathbb{R}^{B, C_3, H_3, W_3}$9.
• Fusion: $C_3=1024$0
• Channel Restoration: $C_3=1024$1, with $C_3=1024$2 a $C_3=1024$3 convolution to 2048 channels.

The variable definitions and parameterization are tightly specified, ensuring implementation fidelity.

4. Module Integration and Downstream Effects

The output of MPFR ($C_3=1024$4) has the same spatial size and final channel dimension as the deep backbone feature ($C_3=1024$5), enabling direct integration with SDCE. The SDCE then proceeds with a two-step transformer-based cascade:

• Block 1: Cross-attention with $C_3=1024$6, $C_3=1024$7.
• Block 2: Self-attention on Block 1's output.

These operations further distill identity-aware features, after which global pooling and an ICG loss are applied to optimize cross-modal feature separation. The MPFR-generated features are thus not endpoint representations, but rather identity-sensitive feature banks for subsequent semantic distillation and final Re-ID embedding formation (Zhang et al., 4 Dec 2025).

5. Pseudocode Specification

A forward pass for MPFR consistent with the paper's implementation detail can be represented as:

$C_3=1024$8

This implementation is specified for reproducibility and accuracy, ensuring that feature alignment, masking, and fusion follow the outlined processing flow (Zhang et al., 4 Dec 2025).

6. Empirical Validation

Systematic ablation on the SYSU-MM01 all-search, single-shot VI-ReID protocol isolates the effect of MPFR:

• Baseline (AGW + Triplet): Rank-1 = 70.21%, mAP = 68.48%
• +MPFR only (Triplet): Rank-1 = 76.22% (+6.01), mAP = 72.66% (+4.18)
• +MPFR only (ICG Loss): Rank-1 = 77.51%, mAP = 74.16%

These results indicate a substantial, isolated performance gain from MPFR integration. Additionally, feature-distribution plots demonstrate reduced intra-class (same ID) distances and increased inter-class separation post-MPFR. Visualizations via Grad-CAM reveal that MPFR shifts focus towards semantically salient body regions and away from backgrounds. Top-10 retrieval results confirm reduction in false matches due to MPFR's effect (Zhang et al., 4 Dec 2025).

7. Conceptual Significance and Role in Cross-Modal Person Re-ID

MPFR is explicitly designed to extract and leverage modality-specific “identity clues” that reside in shallow convolutional responses, particularly color, texture, and thermal patterns that are often diluted in deeper, modality-invariant embeddings. By generating spatial masks for scale-aware fusion, the module harnesses information crucial to discriminative learning in VI-ReID. Its empirical performance and modular, lightweight design demonstrate that gathering and refining multi-scale shallow features significantly enhances cross-modal retrieval tasks without introducing substantial architectural complexity. A plausible implication is that future VI-ReID models may increasingly integrate analogous spatial-scale-aware modules to maximize representation diversity and robustness in the presence of modality gaps (Zhang et al., 4 Dec 2025).