BRST: Cross-Modal Pseudo-Label Filtering

• BRST is a cross-modal pseudo-label filtering method that ensures mutual consistency between visible and infrared clusters in unsupervised re-ID.
• It employs bidirectional matching and a confidence threshold to selectively propagate high-quality pseudo-labels across modalities.
• BRST significantly boosts re-ID performance, achieving double-digit improvements in Rank-1 and mAP on benchmarks like SYSU-MM01 and RegDB.

Searching arXiv for the cited paper and related context on USVI-ReID/HIL. Bidirectional Reverse Selection Transmission (BRST) is a cross-modal pseudo-label filtering and propagation mechanism introduced within the Hierarchical Identity Learning (HIL) framework for unsupervised visible-infrared person re-identification (USVI-ReID) (Shi et al., 15 Sep 2025). In USVI-ReID, the absence of true identity annotations requires training from pseudo-labels generated by clustering, but clustering each modality independently often produces noisy cross-modal associations. BRST addresses this problem by matching both directions between visible instances and infrared coarse-grained centroids, retaining only matches that are mutually consistent and sufficiently confident, and then transmitting these reliable pseudo-labels bidirectionally across modalities. Within HIL, BRST is designed to improve the quality of modality-unified pseudo-labels used by subsequent contrastive objectives, thereby reducing label noise during unsupervised training (Shi et al., 15 Sep 2025).

1. Problem setting and motivation

BRST arises in the setting of USVI-ReID, where the objective is to learn modality-invariant image features from unlabeled cross-modal person datasets by reducing the modality gap while minimizing reliance on costly manual annotations (Shi et al., 15 Sep 2025). Existing methods are described as typically addressing USVI-ReID using cluster-based contrastive learning, representing a person by a single cluster center, but primarily focusing on commonality within each cluster while neglecting finer-grained differences among images. The HIL framework was proposed to address that broader limitation, and BRST was introduced as the component responsible for more reliable cross-modal correspondence construction.

The immediate motivation for BRST is the unreliability of naive cross-modal pseudo-label assignment. In the described training setting, coarse-grained clustering is first performed separately on visible and infrared features, yielding cluster centroids $\{\mathbf{u}^{vc}_i\}$ and $\{\mathbf{u}^{rc}_j\}$. Without cross-modal supervision, one may assign each visible instance to its nearest infrared centroid, or conversely assign each infrared instance to its nearest visible centroid, and use the resulting index as a unified pseudo-label. The stated problem is that many such nearest-neighbor assignments are one-way only, or are too weakly matched, so direct propagation of these assignments injects noise into the learner (Shi et al., 15 Sep 2025).

A common misconception in this setting is that nearest-centroid assignment alone is an adequate surrogate for cross-modal supervision. The formulation of BRST explicitly rejects that assumption: the mechanism was designed because many visible-to-infrared matches are unreliable, and noisy matches can mislead the representation learner. This suggests that BRST should be understood less as a general matching procedure than as a reliability filter for pseudo-label transfer.

2. Core mechanism and algorithmic workflow

BRST filters and selects only those cross-modal matches that are mutually consistent and sufficiently confident, then transmits these high-quality pseudo-labels from one modality to the other in a bidirectional fashion (Shi et al., 15 Sep 2025). The procedure is defined for visible instances against infrared coarse-grained centroids, and then repeated with roles swapped for infrared instances against visible coarse-grained centroids.

For the visible-to-infrared direction, the input consists of visible instance features $\mathcal{F}^v=\{\mathbf{f}^v_k\}_{k=1}^{N_v}$, infrared coarse-grained centroids $\mathcal{U}^{rc}=\{\mathbf{u}^{rc}_i\}_{i=1}^{M_r}$, and a threshold $\gamma\in(0,1]$. The output is a set of unified visible pseudo-labels $\{\hat y^v_k\}$. The algorithm first computes a cosine-similarity matrix between each visible instance and each infrared centroid. For each visible instance $k$, it finds the best-matching infrared cluster $i^*$ and its similarity score $m_r$. It then checks the reverse direction by finding, for that same infrared cluster $i^*$, the maximum similarity $\{\mathbf{u}^{rc}_j\}$0 over all visible instances. The match is accepted only if $\{\mathbf{u}^{rc}_j\}$1; otherwise the visible instance is marked unmatched. The same procedure is repeated for infrared instances to obtain $\{\mathbf{u}^{rc}_j\}$2 (Shi et al., 15 Sep 2025).

The pseudo-code provided for the visible branch is:

$\{\hat y^v_k\}$3

The terminology of the method is directly tied to its operations. “Bidirectional” refers to the visible$\{\mathbf{u}^{rc}_j\}$3infrared and infrared$\{\mathbf{u}^{rc}_j\}$4visible passes. “Reverse selection” refers to the reverse-direction verification of a forward candidate match. “Transmission” refers to the assignment of the accepted cross-modal cluster index as the new pseudo-label for the instance.

3. Mathematical formulation

The BRST formulation is expressed through a sequence of cosine-similarity and max-selection operations (Shi et al., 15 Sep 2025). For visible instance $\{\mathbf{u}^{rc}_j\}$5 and infrared centroid $\{\mathbf{u}^{rc}_j\}$6, the similarity matrix is

$\{\mathbf{u}^{rc}_j\}$7

The forward match for visible instance $\{\mathbf{u}^{rc}_j\}$8 is defined by

$\{\mathbf{u}^{rc}_j\}$9

The reverse match for the selected cluster $\mathcal{F}^v=\{\mathbf{f}^v_k\}_{k=1}^{N_v}$0 is

$\mathcal{F}^v=\{\mathbf{f}^v_k\}_{k=1}^{N_v}$1

The acceptance rule is the bidirectional consistency test

$\mathcal{F}^v=\{\mathbf{f}^v_k\}_{k=1}^{N_v}$2

Under this rule, the output pseudo-label for a visible instance is either the accepted infrared cluster index $\mathcal{F}^v=\{\mathbf{f}^v_k\}_{k=1}^{N_v}$3 or no label for that round. The same mathematical structure is then applied symmetrically in the infrared-to-visible direction by swapping the roles of $\mathcal{F}^v=\{\mathbf{f}^v_k\}_{k=1}^{N_v}$4 and $\mathcal{F}^v=\{\mathbf{f}^v_k\}_{k=1}^{N_v}$5.

The formulation makes the confidence criterion explicit. The mechanism does not accept a candidate merely because it is the best for a particular instance; it also requires that the instance’s best score exceed a fraction $\mathcal{F}^v=\{\mathbf{f}^v_k\}_{k=1}^{N_v}$6 of the cluster’s own top score. A plausible implication is that BRST operationalizes confidence through relative rather than absolute similarity, which is consistent with its stated purpose of suppressing weak or one-way matches.

4. Selection and transmission strategy

The selection strategy combines bidirectional matching with a similarity-ratio threshold (Shi et al., 15 Sep 2025). Bidirectional matching ensures that if visible instance $\mathcal{F}^v=\{\mathbf{f}^v_k\}_{k=1}^{N_v}$7 claims infrared cluster $\mathcal{F}^v=\{\mathbf{f}^v_k\}_{k=1}^{N_v}$8 as its best match, then reverse-examining cluster $\mathcal{F}^v=\{\mathbf{f}^v_k\}_{k=1}^{N_v}$9 also claims that visible instance as its best match. The stated effect is to cut out many one-way false positives. The ratio test $\mathcal{U}^{rc}=\{\mathbf{u}^{rc}_i\}_{i=1}^{M_r}$0 further requires that the forward best similarity be sufficiently strong relative to the cluster’s own best visible match.

The threshold $\mathcal{U}^{rc}=\{\mathbf{u}^{rc}_i\}_{i=1}^{M_r}$1 is set via cross-validation. The provided examples are $\mathcal{U}^{rc}=\{\mathbf{u}^{rc}_i\}_{i=1}^{M_r}$2 on SYSU-MM01 and $\mathcal{U}^{rc}=\{\mathbf{u}^{rc}_i\}_{i=1}^{M_r}$3 on RegDB. Instances satisfying both conditions are transmitted, meaning that the infrared cluster index becomes the new pseudo-label for that visible instance; all other instances are left unlabeled in that round. The same logic applies in the opposite direction when infrared instances are processed against visible centroids.

This design distinguishes BRST from unconditional label propagation. The mechanism does not attempt to maximize coverage over all samples. Instead, it prioritizes high-confidence transmission and explicitly allows unmatched instances to remain unlabeled. The paper states that this transmission of only high-confidence labels reduces noise in subsequent contrastive learning. This suggests that BRST trades pseudo-label quantity for pseudo-label precision, with the intent of improving downstream representation learning rather than enforcing exhaustive assignment.

5. Position within Hierarchical Identity Learning

Within HIL, BRST is one of three named components: hierarchical pseudo-label generation, Multi-Center Contrastive Learning (MCCL), and BRST itself (Shi et al., 15 Sep 2025). HIL is motivated by the observation that each coarse-grained cluster may contain several smaller sub-clusters reflecting fine-grained variations among images; accordingly, the framework generates multiple memories for each existing coarse-grained cluster via a secondary clustering. MCCL then refines representations for enhancing intra-modal clustering and minimizing cross-modal discrepancies.

BRST is integrated into this pipeline as the module that establishes reliable cross-modal correspondences. In the overall HIL pipeline, identified as Algorithm 1 in the paper, BRST is invoked once per epoch, alternating between computing $\mathcal{U}^{rc}=\{\mathbf{u}^{rc}_i\}_{i=1}^{M_r}$4 and $\mathcal{U}^{rc}=\{\mathbf{u}^{rc}_i\}_{i=1}^{M_r}$5. These modality-unified pseudo-labels are then used alongside the hierarchical pseudo-labels in several contrastive losses. The identity loss $\mathcal{U}^{rc}=\{\mathbf{u}^{rc}_i\}_{i=1}^{M_r}$6 uses pseudo-labels to pull instances toward their assigned cluster center. The neighbor contrastive loss $\mathcal{U}^{rc}=\{\mathbf{u}^{rc}_i\}_{i=1}^{M_r}$7 exploits instance-to-instance relationships. MCCL, through $\mathcal{U}^{rc}=\{\mathbf{u}^{rc}_i\}_{i=1}^{M_r}$8, uses fine-grained and coarse-grained centers as anchors (Shi et al., 15 Sep 2025).

The stated significance of BRST inside HIL is that it improves the reliability of cross-modal label assignments and thereby boosts the quality of the contrastive signals used by these losses. Because the pseudo-labels produced by BRST are modality-unified, they provide a bridge between modality-specific clustering results and the cross-modal objectives of the overall framework. A plausible implication is that BRST serves as the linkage layer between hierarchical structure discovery and cross-modal metric alignment.

6. Empirical evidence and reported impact

The paper reports extensive experiments on the SYSU-MM01 and RegDB datasets and states that the proposed method outperforms existing approaches (Shi et al., 15 Sep 2025). More specifically, the ablation study isolates the contribution of BRST on two benchmarks, using a baseline HIL configuration with identity and neighbor losses only.

Benchmark	Setting	Result
SYSU-MM01	Baseline (no BRST)	$\mathcal{U}^{rc}=\{\mathbf{u}^{rc}_i\}_{i=1}^{M_r}$9, $\gamma\in(0,1]$0
SYSU-MM01	+ BRST	$\gamma\in(0,1]$1, $\gamma\in(0,1]$2
RegDB	Baseline	$\gamma\in(0,1]$3, $\gamma\in(0,1]$4
RegDB	+ BRST	$\gamma\in(0,1]$5, $\gamma\in(0,1]$6

The reported gains are $\gamma\in(0,1]$7 Rank-1 and $\gamma\in(0,1]$8 mAP on SYSU-MM01 (All Search), and $\gamma\in(0,1]$9 Rank-1 and $\{\hat y^v_k\}$0 mAP on RegDB (Visible$\{\hat y^v_k\}$1Infrared) (Shi et al., 15 Sep 2025). The paper further states that when BRST is combined with MCCL, additional gains are obtained relative to the BRST-only augmentation, and that in a head-to-head comparison with other cross-modal association schemes, replacing OPTM or CRA with BRST on the same baseline produces the highest mAP and Rank-1.

The mechanism is also characterized as lightweight, with $\{\hat y^v_k\}$2 time complexity. The reported interpretation is that by enforcing mutual top-matching and a similarity-ratio check, BRST reduces label noise during unsupervised training and leads to double-digit improvements in both Rank-1 and mAP on standard USVI-ReID benchmarks. Since these gains are reported in the context of an ablation study, they specifically support the contribution of BRST rather than only the aggregate effect of the full HIL framework.

7. Interpretation, scope, and related considerations

BRST should be understood as a pseudo-label reliability mechanism for unsupervised cross-modal representation learning rather than as a generic clustering algorithm (Shi et al., 15 Sep 2025). Its inputs depend on separately clustered modality-specific features and coarse-grained centroids, and its outputs are modality-unified pseudo-labels for instances that satisfy mutual and confidence-based constraints. The method therefore operates after clustering but before the contrastive losses consume the labels.

Its strengths, as explicitly documented, are the filtering of unreliable cross-modal associations, the bidirectional enforcement of consistency, and the reduction of noise in subsequent contrastive learning. Its selectivity is equally central: instances failing the acceptance test are left unlabeled in that round. This suggests a deliberate asymmetry between reliability and completeness, where conservative transmission is preferred to dense but noisy cross-modal supervision.

Within the terminology of the paper, BRST complements MCCL rather than replacing it. MCCL is responsible for refining representations with fine-grained and coarse-grained centers as anchors, whereas BRST governs which cross-modal pseudo-labels are sufficiently trustworthy to be transmitted. In that sense, HIL combines hierarchical structure modeling, contrastive representation refinement, and cross-modal pseudo-label verification into a single training pipeline (Shi et al., 15 Sep 2025).

The primary source for BRST is the paper "Hierarchical Identity Learning for Unsupervised Visible-Infrared Person Re-Identification" (Shi et al., 15 Sep 2025), whose source code is reported as available at https://github.com/haonanshi0125/HIL.