Pseudo-label Consistency Score (PCS)

• Pseudo-label Consistency Score (PCS) is a metric family that quantifies pseudo-label stability via measures like count-gap and classification confidence.
• PCS is computed by aggregating repeated predictions or transformed inputs to assess temporal, spatial, and discriminatory consistency in noisy or weakly supervised data.
• PCS methods are applied to refine labels in tasks such as object localization, hyperspectral classification, and LLM annotation, enhancing training reliability and performance.

Searching arXiv for papers on pseudo-label consistency score and closely related pseudo-label consistency mechanisms. Pseudo-label Consistency Score (PCS) denotes a family of consistency- or confidence-oriented quantities used to assess the reliability of pseudo-labels under weak, noisy, or indirect supervision. The term is not used uniformly across the literature: some papers define an explicit scalar called PCS, whereas others introduce closely related mechanisms under names such as a “pseudo-label confidence evaluation method based on classification discrimination,” “Count-Gap (CG),” or spatial pseudo-label enhancement. Across these variants, the common function is to distinguish pseudo-labels that are stable, discriminative, or structurally coherent from those that are noisy or ambiguous, and to use that distinction to gate refinement, sample selection, or loss design (Liu et al., 19 Sep 2025, Sun et al., 2022, Qiu et al., 26 Jan 2026, Gabrielyan et al., 12 Feb 2026).

1. Terminological scope and conceptual definition

PCS is defined explicitly in some settings and only implicitly in others. In weakly supervised object localization, the relevant paper states that it does not explicitly use the term “PCS”, but introduces a stage-two pseudo-label confidence evaluation method based on classification discrimination; in effect, that confidence score functions as PCS for pseudo bounding boxes (Sun et al., 2022). In semi-supervised hyperspectral image classification, the paper likewise states that it does not explicitly define a metric named “Pseudo-label Consistency Score (PCS)”, but identifies Count-Gap (CG) together with Dynamic History-Fused Prediction (DHP) and Adaptive Tripartite Sample Categorization (ATSC) as the closest analogue (Qiu et al., 26 Jan 2026). In scribble-supervised medical segmentation, the paper explicitly does not define a standalone metric named “Pseudo-label Consistency Score (PCS),” yet centers its method on improving the reliability, spatial consistency, and quality of pseudo-labels through hierarchical structural refinement (Gabrielyan et al., 12 Feb 2026).

Where PCS is explicitly named, it is typically framed as a score of prediction stability. In noisy correspondence learning for cross-modal retrieval, Pseudo-label Consistency Score (PCS) is defined to quantify prediction stability and reflect the model’s level of understanding of an image (Liu et al., 19 Sep 2025). In zero-shot LLM annotation, the acronym PCS refers instead to Perceived Confidence Score, a black-box confidence estimation and label-selection method that measures consistency of labels across semantically equivalent input mutations (Salimian et al., 11 Feb 2025). This suggests that PCS is best treated not as a single canonical metric, but as a broader design pattern for converting pseudo-label stability into an operational signal for downstream training decisions.

A plausible implication is that the literature organizes pseudo-label reliability along at least three axes: classification discrimination, temporal stability, and spatial coherence. These axes recur even when the score is unnamed.

2. Formalizations of PCS and PCS-like quantities

The most direct scalar PCS formulation in the provided literature appears in cross-modal noisy correspondence learning. For an image $I_i$, a pseudo-classifier produces a prediction vector $p_i^c = C(f(I_i^c))$, from which the pseudo-label is

$\hat{p}_i^c = \arg\max(p_i^c).$

Across multiple epochs or forward passes, the model accumulates prediction counts

$Q_i = [n_1, n_2, \ldots, n_c].$

PCS is then defined as the gap between the most frequently predicted class and the second most frequent class: $\mathrm{PCS}_i = n_{\max} - n_{\mathrm{second}}.$ Larger PCS indicates higher stability of the pseudo-label over time (Liu et al., 19 Sep 2025).

A closely related temporal-consistency statistic appears in hyperspectral classification as Count-Gap (CG): $CG(u_i) = f_{\max}(u_i) - f_{\text{second}}(u_i),$ where $f_{\max}(u_i)$ is the count of the most frequently predicted class for $u_i$, and $f_{\text{second}}(u_i)$ is the count of the second most frequently predicted class (Qiu et al., 26 Jan 2026). The historical counts come from a queue

$$Q_i^{(t)} = \{ \text{cnt}_i^{(t)}(C_1), \text{cnt}_i^{(t)}(C_2), \dots, \text{cnt}_i^{(t)}(C_K) \},$$

and are fused with current predictions through

$p_i^c = C(f(I_i^c))$0

with confidence given by

$p_i^c = C(f(I_i^c))$1

Here, CG functions as the paper’s clearest consistency score, while $p_i^c = C(f(I_i^c))$2 provides a complementary confidence signal (Qiu et al., 26 Jan 2026).

In weakly supervised object localization, the PCS-like quantity is based on the maximum classification probability of a cropped pseudo box image $p_i^c = C(f(I_i^c))$3: $p_i^c = C(f(I_i^c))$4 and a pseudo box is retained if

$p_i^c = C(f(I_i^c))$5

The high-quality indicator is written as

$p_i^c = C(f(I_i^c))$6

with $p_i^c = C(f(I_i^c))$7 in the main experiments and $p_i^c = C(f(I_i^c))$8 in ablations (Sun et al., 2022).

In zero-shot LLM annotation, Perceived Confidence Score (PCS) is computed from label frequencies across one original input and three Metamorphic Relation (MR) variants. The paper first gives the unweighted intuition—for example, if the four outputs are biased, biased, biased, unbiased, then PCS for biased is $p_i^c = C(f(I_i^c))$9. It then defines a weighted label score per LLM as

$\hat{p}_i^c = \arg\max(p_i^c).$0

and aggregates across LLMs as

$\hat{p}_i^c = \arg\max(p_i^c).$1

The final label is chosen by thresholding these scores, with fallback to majority voting (Salimian et al., 11 Feb 2025).

In scribble-supervised segmentation, the paper does not define a scalar PCS, but provides a structured pseudo-label enhancement mechanism. The original pseudo-label is

$\hat{p}_i^c = \arg\max(p_i^c).$2

scribbles are spread hierarchically as

$\hat{p}_i^c = \arg\max(p_i^c).$3

then refined with background expansion

$\hat{p}_i^c = \arg\max(p_i^c).$4

and finally mixed with the pseudo-label under a mask: $\hat{p}_i^c = \arg\max(p_i^c).$5 This suggests a structural notion of PCS in which consistency is expressed through hierarchical region agreement rather than a single scalar threshold (Gabrielyan et al., 12 Feb 2026).

3. Computational mechanisms

Despite differing formulations, PCS-like methods share a common computational structure: repeated or auxiliary predictions are collected, converted into a reliability statistic, and then used to filter or refine supervision.

In object localization, computation proceeds from a raw-image localization prediction

$\hat{p}_i^c = \arg\max(p_i^c).$6

followed by cropping the corresponding image region $\hat{p}_i^c = \arg\max(p_i^c).$7, classifying that crop to obtain $\hat{p}_i^c = \arg\max(p_i^c).$8, and taking

$\hat{p}_i^c = \arg\max(p_i^c).$9

Only crops whose PCS exceeds $Q_i = [n_1, n_2, \ldots, n_c].$0 contribute to refinement (Sun et al., 2022).

In cross-modal retrieval, the prediction history of a pseudo-classifier is the key input. The algorithm does not use a one-shot confidence but accumulates the frequencies of predicted pseudo-categories over multiple epochs or forward passes, producing $Q_i = [n_1, n_2, \ldots, n_c].$1, from which PCS is the top-two count gap (Liu et al., 19 Sep 2025). A plausible implication is that this formulation targets semantic stability rather than instantaneous certainty.

In hyperspectral classification, the mechanism is similar but embedded in a longer temporal pipeline. For each unlabeled sample, a historical prediction queue is maintained, its length grows during training, and the queue statistics are converted into a historical distribution $Q_i = [n_1, n_2, \ldots, n_c].$2. The current and historical distributions are fused by DHP, then both fused confidence $Q_i = [n_1, n_2, \ldots, n_c].$3 and consistency $Q_i = [n_1, n_2, \ldots, n_c].$4 are computed and passed to ATSC (Qiu et al., 26 Jan 2026). The paper explicitly describes this as making pseudo-labels smoother and less volatile over time.

In zero-shot LLM annotation, the repeated evidence comes not from training epochs but from semantically equivalent textual perturbations. The three MRs are MR1 - Active to Passive Transformation, MR2 - Double Negation Transformation, and MR3 - Synonym Replacement (Salimian et al., 11 Feb 2025). Each original input and its mutated variants are labeled by one or more LLMs, and PCS is computed from the resulting frequency pattern. Because the method is designed for black-box LLMs, it uses only observed labels rather than internal probabilities (Salimian et al., 11 Feb 2025).

In scribble-supervised segmentation, the mechanism relies on image structure. Each 2D slice is partitioned into a hierarchy of semantically coherent regions using watershed/waterfall transforms, and labels are spread inside the coarse layers, then inside the finer layers of the partition hierarchy (Gabrielyan et al., 12 Feb 2026). If a region contains only one class candidate, that class is assigned; if there are conflicting candidates, the region remains unlabeled. This is a region-constrained consistency mechanism that reduces leakage across uncertain boundaries.

4. Roles in training and sample refinement

PCS-like quantities are not merely descriptive; they determine which pseudo-labels influence optimization and how.

In weakly supervised object localization, the stage-two regression consistency loss is

$Q_i = [n_1, n_2, \ldots, n_c].$5

PCS therefore acts as a gate: only pseudo boxes that are sufficiently class-discriminative participate in consistency learning (Sun et al., 2022). The paper explicitly states that low-quality pseudo-labels can degrade performance.

In cross-modal retrieval, PCS is used after a confidence-based clean/noisy split to further divide noisy pairs into

$Q_i = [n_1, n_2, \ldots, n_c].$6

and

$Q_i = [n_1, n_2, \ldots, n_c].$7

The refinable subset $Q_i = [n_1, n_2, \ldots, n_c].$8 is repaired by caption replacement, whereas the ambiguous subset $Q_i = [n_1, n_2, \ldots, n_c].$9 is trained with generalized cross-entropy and other robust objectives under Adaptive Pair Optimization (APO) (Liu et al., 19 Sep 2025). PCS thus enables a three-way view of the data—clean, ambiguous noisy, and refinable noisy—rather than a binary clean/noisy split.

In hyperspectral classification, confidence and consistency jointly drive Adaptive Tripartite Sample Categorization (ATSC). The paper defines:

• Easy samples:

$\mathrm{PCS}_i = n_{\max} - n_{\mathrm{second}}.$0

• Ambiguous samples:

$\mathrm{PCS}_i = n_{\max} - n_{\mathrm{second}}.$1

• Hard samples:

$\mathrm{PCS}_i = n_{\max} - n_{\mathrm{second}}.$2

Easy samples are used with hard pseudo-labels, ambiguous samples with soft predictions and KL consistency loss, and hard samples are discarded temporarily (Qiu et al., 26 Jan 2026). This is an explicit example of PCS-like information governing hierarchical sample utilization.

In scribble-supervised segmentation, the enhanced pseudo-labels directly define the loss

$\mathrm{PCS}_i = n_{\max} - n_{\mathrm{second}}.$3

The paper states that the enhanced scribble information overrides the pseudo-label only on enhanced scribble pixels and only during an early portion of training (Gabrielyan et al., 12 Feb 2026). This suggests that PCS-like refinement can also be temporally scheduled, not merely thresholded.

5. Empirical evidence and reported effects

The literature consistently reports that pseudo-label filtering or enhancement based on consistency improves downstream performance, although the measured outcome varies by task.

In object localization, the threshold ablation on CUB-200-2011 shows that as $\mathrm{PCS}_i = n_{\max} - n_{\mathrm{second}}.$4 increases, retained pseudo-labels become fewer but cleaner. The reported LCHP-R GT-Known scores rise from 88.36 at $\mathrm{PCS}_i = n_{\max} - n_{\mathrm{second}}.$5 to 91.18 at $\mathrm{PCS}_i = n_{\max} - n_{\mathrm{second}}.$6, 92.80 at $\mathrm{PCS}_i = n_{\max} - n_{\mathrm{second}}.$7, 93.98 at $\mathrm{PCS}_i = n_{\max} - n_{\mathrm{second}}.$8, and 94.36 at $\mathrm{PCS}_i = n_{\max} - n_{\mathrm{second}}.$9; by contrast, $CG(u_i) = f_{\max}(u_i) - f_{\text{second}}(u_i),$0 keeps all boxes, but refinement fails (Sun et al., 2022). The same paper reports that LCHP-R improves over LCHP-I on all three benchmark datasets, including 73.61 Top-1 / 87.12 GT-Known to 80.39 Top-1 / 94.36 GT-Known on CUB-200-2011 (Sun et al., 2022).

In hyperspectral classification, ablations show that removing DHP or ATSC lowers performance. Table IV reports, for example, on PaviaU an OA of 95.21 for the full method versus 94.20 w/o DHP, and on Houston2013 an OA of 89.77 versus 89.38 w/o DHP (Qiu et al., 26 Jan 2026). The paper also states that removing ATSC causes performance drops by about 1% OA/AA on PaviaU and Houston2013. These gains are described as evidence that temporal smoothing and consistency-aware sample categorization improve pseudo-label reliability (Qiu et al., 26 Jan 2026).

In scribble-supervised cardiac MRI segmentation, PLESS improves all four scribble-supervised algorithms overall on ACDC, with notable gains in boundary metrics. The paper reports that DMPLS improves from 9.37 to 7.10 HD95 and from 2.53 to 2.01 ASD, while ScribbleVS improves from 9.77 to 6.39 HD95 and from 2.71 to 1.66 ASD (Gabrielyan et al., 12 Feb 2026). On MSCMRseg, the paper states that PLESS also consistently improves HD95 and ASD for most methods, even when DSC does not always increase. The authors interpret this as evidence that “PLESS consistently improves boundary accuracy and surface agreement, leading to more reliable segmentations across multiple architectures.” They further note that qualitative examples show PLESS fixing false positives, reducing hallucinated classes, and better handling ambiguous slices (Gabrielyan et al., 12 Feb 2026).

In noisy correspondence learning, the paper reports that PCSR achieves the best or near-best retrieval performance under multiple noise levels on Flickr30K, MSCOCO, and CC152K, with especially clear gains under high noise (Liu et al., 19 Sep 2025). The ablation using subsets $CG(u_i) = f_{\max}(u_i) - f_{\text{second}}(u_i),$1, $CG(u_i) = f_{\max}(u_i) - f_{\text{second}}(u_i),$2, and $CG(u_i) = f_{\max}(u_i) - f_{\text{second}}(u_i),$3 reports Rsum = 480.5 when all three subsets are used, and states that removing either ambiguous or refinable data hurts performance, with removing refinable data causing a larger drop (Liu et al., 19 Sep 2025). Figure-based analysis further shows that semantically simpler images tend to have higher PCS, while complex images tend to have lower PCS (Liu et al., 19 Sep 2025).

In zero-shot LLM annotation, PCS significantly improves zero-shot accuracy for Llama-3-8B-Instruct (4.96%), Mistral-7B-Instruct-v0.3 (10.52%), and Gemma-2-9b-it (9.39%), while combining all three models yields a 7.75% gain over majority voting (Salimian et al., 11 Feb 2025). The paper also reports average gains of 6.57% over majority voting when two LLMs are used (Salimian et al., 11 Feb 2025). These results support the broader claim that consistency under semantically equivalent perturbations can serve as a useful confidence proxy when internal model probabilities are unavailable.

6. Common principles, misconceptions, and limitations

A recurrent misconception is that PCS always denotes a standardized scalar metric. The cited literature shows the opposite. Some papers explicitly define PCS as a count-gap statistic over pseudo-label histories (Liu et al., 19 Sep 2025); others use maximum class probability as a PCS-equivalent confidence gate (Sun et al., 2022); others substitute related measures such as CG and fused confidence (Qiu et al., 26 Jan 2026); still others encode consistency structurally through pseudo-label enhancement without introducing any scalar called PCS (Gabrielyan et al., 12 Feb 2026). The term is therefore method-dependent.

Another misconception is that more propagation or broader pseudo-label coverage is always beneficial. The segmentation ablation with progressive variants “enh,” “enh+bg,” and “enh+bg+prop” shows that moderate enhancement is best, while full propagation often hurts because “aggressive scribble spreading at full tolerance can introduce noise” (Gabrielyan et al., 12 Feb 2026). This indicates that consistency control is not equivalent to maximal label expansion.

A further limitation is that PCS often depends on assumptions about invariance or stability. In zero-shot LLM annotation, PCS assumes that the Metamorphic Relations preserve the correct label; the paper explicitly notes the assumption of semantic invariance and warns that if an MR changes meaning subtly, PCS could be misled (Salimian et al., 11 Feb 2025). The same paper also notes dependence on predefined MRs and limited dataset coverage as limitations (Salimian et al., 11 Feb 2025). In object localization, using all pseudo-labels without thresholding causes refinement failure, showing that a poorly calibrated PCS gate can be harmful (Sun et al., 2022).

A plausible implication is that PCS methods are most effective when the source of variation is task-aligned: temporal variation for pseudo-label stability, augmentation variation for consistency under transformations, and spatial partition variation for anatomical coherence.

7. Relationship to adjacent research directions

PCS is closely related to, but distinct from, ordinary confidence scoring, consistency regularization, and pseudo-label generation. In object localization, the paper explicitly compares its pseudo-label filtering to confidence filtering in FixMatch, but adapted to localization (Sun et al., 2022). In hyperspectral classification, ATSC combines both confidence and consistency measures, implying that current predictive certainty and temporal agreement provide nonredundant information (Qiu et al., 26 Jan 2026). In cross-modal retrieval, PCS is introduced precisely because coarse clean/noisy categorization overlooks structural diversity within noisy samples (Liu et al., 19 Sep 2025).

The segmentation literature adds a different perspective: pseudo-label quality is not only a matter of prediction stability but also of spatial plausibility. PLESS is described as not just another pseudo-label generator; it is a pseudo-label refiner that injects spatial anatomy-aware priors into the target labels (Gabrielyan et al., 12 Feb 2026). This extends the PCS idea from temporal or probabilistic agreement to anatomical and boundary-aware coherence.

The broader methodological pattern can therefore be summarized as follows. First, a model or auxiliary process produces tentative pseudo-labels. Second, a reliability statistic—explicit PCS, CG, maximum class probability, or structural enhancement—is computed from repeated predictions, fused history, crop discrimination, or image regions. Third, training logic is conditioned on that statistic through thresholding, sample triage, masked replacement, or specialized loss selection. The repeated empirical finding is that pseudo-labels judged stable, discriminative, or spatially coherent are more useful supervisory signals than raw pseudo-labels (Liu et al., 19 Sep 2025, Sun et al., 2022, Qiu et al., 26 Jan 2026, Gabrielyan et al., 12 Feb 2026).

In this sense, PCS is less a single metric than a recurring principle in weakly supervised and semi-supervised learning: pseudo-labels should be trusted in proportion to their demonstrable consistency.