ICRE Network for VI-ReID Identity Enhancement

• ICRE Network is a deep neural framework that refines identity cues by leveraging both modality-invariant and discriminative modality-specific features.
• It incorporates a modular architecture with MPFR for multi-scale feature fusion, SDCE for transformer-based enhancement, and an ICG loss for optimizing cross-modal clustering.
• Empirical evaluations on SYSU-MM01 show that integrating MPFR and ICG leads to significant performance gains, with up to a 6% increase in Rank-1 accuracy.

The Identity Clue Refinement and Enhancement (ICRE) Network is a deep neural framework designed to address the domain gap and modality discrepancy problem in Visible-Infrared Person Re-Identification (VI-ReID), where the goal is to match pedestrian images captured under visible and infrared spectrums. Unlike conventional approaches focused solely on modality-invariant features, ICRE incorporates both modality-invariant and discriminative modality-specific knowledge to enhance cross-modal person matching performance through explicit refinement and enhancement of identity cues (Zhang et al., 4 Dec 2025).

1. Motivation and Conceptual Underpinnings

VI-ReID poses unique challenges due to the significant discrepancy between visible and infrared domains, mainly stemming from differences in appearance caused by environmental and sensor factors. Mainstream methods concentrate on learning modality-invariant embeddings, optimizing only for the discriminative features shared across domains, often at the expense of rich identity-related cues unique to each modality. ICRE is introduced to bridge this gap by explicitly mining, refining, and transferring modality-specific but identity-relevant information into the learned representations, enhancing the discriminative capacity of deep features for cross-modal retrieval (Zhang et al., 4 Dec 2025).

2. Architectural Design: Modules and Data Flow

The ICRE network consists of three critical and structurally independent modules: Multi-Perception Feature Refinement (MPFR), Semantic Distillation Cascade Enhancement (SDCE), and the Identity Clues Guided (ICG) Loss, constructed atop a shared-branch ResNet-50 backbone.

2.1 Multi-Perception Feature Refinement (MPFR)

MPFR receives three shallow feature maps from designated ResNet-50 stages:

• $f_{\ell}$: stage-2 output, shape $256 \times 48 \times 24$
• $f_{m}$: stage-3 output, shape $512 \times 24 \times 12$
• $f_{h}$: stage-4 output, shape $1024 \times 12 \times 6$

These are individually mapped via ConvBlock modules (convolution, batch normalization, ReLU activation) to a unified dimensionality $1024 \times 12 \times 6$ through:

• $k_{\ell}=3$, $s_{\ell}=4$, $p_{\ell}=1$
• $k_{m}=3$, $s_{m}=2$, $p_{m}=1$
• $k_{h}=1$, $s_{h}=1$, $p_{h}=0$

Each aligned feature $\hat{f}_i$ undergoes spatial summarization:

• Channel-wise average and max pooling produce a 2-channel map $M_i \in \mathbb{R}^{2 \times 12 \times 6}$.

Parallel 3×3 dilated convolutions (dilations 1, 2, 3) are applied and summed:

• $$M'_i = \varphi^{d=1}(M_i) + \varphi^{d=2}(M_i) + \varphi^{d=3}(M_i)$$

Masks from each scale ($M'_{\ell}$, $M'_m$, $M'_h$) are stacked and normalized by softmax along the scale axis at each spatial location, yielding three weighting masks whose sum is unity at each $(u,v)$. Masked features are fused:

• $$F = M''_{\ell} \odot \hat{f}_{\ell} + M''_m \odot \hat{f}_m + M''_h \odot \hat{f}_h$$
• The fused tensor passes through a 1×1 ConvBlock, producing the output $\tilde{f}$.

Forward-Pass Pseudocode for MPFR

def MPFR(f_l, f_m, f_h): hat_l = ConvBlock(kernel=3, stride=4, out_ch=C4)(f_l) hat_m = ConvBlock(kernel=3, stride=2, out_ch=C4)(f_m) hat_h = ConvBlock(kernel=1, stride=1, out_ch=C4)(f_h) def make_mask(X): avg_map = X.mean(dim=1, keepdim=True) max_map = X.max(dim=1, keepdim=True)[0] M = torch.cat([avg_map, max_map], dim=1) m1 = DilatedConv(2->1, kernel=3, dilation=1)(M) m2 = DilatedConv(2->1, kernel=3, dilation=2)(M) m3 = DilatedConv(2->1, kernel=3, dilation=3)(M) return m1 + m2 + m3 m_l = make_mask(hat_l) m_m = make_mask(hat_m) m_h = make_mask(hat_h) M_stack = torch.cat([m_l, m_m, m_h], dim=1) weights = torch.softmax(M_stack, dim=1) w_l, w_m, w_h = weights[:,0], weights[:,1], weights[:,2] F = w_l.unsqueeze(1)*hat_l + w_m.unsqueeze(1)*hat_m + w_h.unsqueeze(1)*hat_h out = ConvBlock(kernel=1, stride=1, out_ch=C4)(F) return out

Here, $C_4=1024$, matching all spatial resolutions to $12 \times 6$.

2.2 Semantic Distillation Cascade Enhancement (SDCE)

The feature $\tilde{f}$ output by MPFR is passed as key and value inputs to the first transformer block in the SDCE module, while the original deep feature $f_h$ serves as the query. The resulting cross-attended feature is subsequently refined via a self-attention block. SDCE's output supersedes or is combined with the original deep feature before global pooling.

2.3 Identity Clues Guided (ICG) Loss

After global GeM pooling, the SDCE feature is input to both a classification head for cross-entropy (ID) loss and the ICG loss. The ICG loss operates by reducing the distance between each sample's feature $f$ and its class center $c_{y_i}^{\bar z}$ from the opposite modality, while simultaneously maximizing the margin to centers of all other classes. Since $\tilde{f}$ directly affects the final pooled embedding, the ICG loss supervises MPFR to yield features optimized for cross-modal intra-class compactness and inter-class separability.

3. Mathematical Operations in MPFR

The MPFR's steps are defined as follows:

• Feature Alignment: $\hat{f}_i = \text{ConvBlock}_i(f_i)$, $i \in \{\ell, m, h\}$
• Spatial Summary: For spatial position $(u,v)$,
  • $$A_i(u, v) = \frac{1}{C_4} \sum_{c=1}^{C_4} \hat{f}_i(c, u, v)$$ (average)
  • $M_i(u,v) = \max_{c} \hat{f}_i(c, u, v)$ (max)
  • Stack as $$M_i = [A_i; M_i] \in \mathbb{R}^{2 \times 12 \times 6}$$
• Mask Generation: $M'_i = \sum_{d \in \{1,2,3\}} \varphi_{3,d}(M_i)$, where $\varphi_{3,d}$ denotes 2$\to$1 channel 3×3 convolutions with dilation $d$
• Mask Weighting: For each $(u,v)$, $$M''_i(u,v) = \frac{\exp(M'_i(u,v))}{\sum_{j \in \{\ell,m,h\}} \exp(M'_j(u,v))}$$
• Feature Fusion: $$F = \sum_{i \in \{\ell,m,h\}} M''_i \odot \hat{f}_i$$; $$\tilde{f} = \mathrm{ReLU}(\mathrm{BN}(W_0 \ast F + b_0))$$

4. Integration and Training Workflow

The output map $\tilde{f}$ from MPFR is deeply integrated into the SDCE transformer-based enhancement module and the global pooling pipeline. It injects adaptive spatially-weighted, cross-scale identity features that are distilled via cross-attention into the deepest stage representation. The SDCE's concatenated or replaced features are then supervised by the dual loss regime (ID loss, ICG loss), encouraging both discrimination and cross-modal alignment.

5. Empirical Validation and Ablation Analyses

Ablation experiments validate the critical role of MPFR. On the SYSU-MM01 dataset (Single-Shot, All-Search protocol):

• Baseline (AGW+triplet): $R_1=70.21\%$, mAP=$68.48\%$
• Replacing triplet with ICG loss (no MPFR): $R_1=72.03\%$, mAP=$69.54\%$
• Add MPFR (with triplet): $R_1 = 76.22\%$, mAP=$72.66\%$ ($+6.0\%$ $R_1$)
• Add MPFR + ICG: $R_1 = 77.51\%$, mAP=$74.16\%$ ($+5.5\%$ $R_1$ vs ICG alone)

Integrating SDCE on top of MPFR further increases performance, reaching $R_1 = 80.41\%$ with ICG. The data demonstrates that MPFR alone yields significant improvement in cross-modal discrimination (~$+6\%$ $R_1$), as evidenced by improved inter-class margin and intra-class cohesion visualized via distribution and t-SNE analyses (Zhang et al., 4 Dec 2025).

6. Functional Significance and Design Implications

The MPFR module is a lightweight and purely shared-branch design that (1) aggregates multi-scale shallow features, (2) dynamically spatially weights them via learned masks, and (3) injects modality-specific identity cues into the holistic representation. This architecture directly addresses the inability of prior modality-invariant methods to utilize discriminative, shallow, modality-dependent information. The ICRE paradigm demonstrates that explicit incorporation and enhancement of identity clues from shallow feature spaces is essential for narrowing the visible–infrared domain gap and achieving state-of-the-art cross-modal person re-identification accuracy (Zhang et al., 4 Dec 2025).