Papers
Topics
Authors
Recent
Search
2000 character limit reached

Class-Based Label Replacement

Updated 10 July 2026
  • Class-based label replacement is a process that substitutes, refines, or reconstructs original class annotations using contextual, class-conditioned evidence to address noise and underexploited semantics.
  • It spans diverse applications such as multi-label augmentation in remote sensing, adversarial substitution in vertical federated learning, and semantic augmentation in few-shot text classification and graph analysis.
  • Empirical results demonstrate significant performance boosts and robustness improvements, with tailored techniques achieving higher mAP, accuracy, and F1 scores across various benchmarks.

Class-based label replacement denotes a family of operations in which the original class annotation is not used verbatim but is instead updated, substituted, reconstructed, or injected through class-conditioned evidence. In the literature, this appears in several distinct forms: recomputing a mixed multi-label vector from propagated pixel-level class maps after CutMix in remote sensing (Burgert et al., 2024), replacing encrypted gradient messages so poisoned samples train toward an attacker-chosen target class in black-boxed vertical federated learning (Liu et al., 2021), embedding class labels into support sentences through a natural-language template in few-shot text classification (Hou et al., 2024), propagating neighboring labels inside a GCN by masking self-loops on label features (Bellei et al., 2021), reconstructing a probabilistic label matrix by inverse tri-NMF (Satoh, 12 Oct 2025), and inferring unseen labels through a class label autoencoder in zero-shot learning (Lin et al., 2018). Related work also shows that some settings are more precisely described as class-informed label promotion or class-aware label cleaning rather than literal replacement (Zhang et al., 2024, Huang et al., 2022).

1. Conceptual scope and operational forms

Across these works, the replaced object is not always the same. In some methods, the target of replacement is the training label itself: the multi-label vector of a CutMix image is recomputed from propagated class-position maps, and the true label of a poisoned sample in VFL is effectively replaced by an attacker-chosen target class through gradient manipulation. In other methods, the replaced object is a representation of class information rather than the final supervisory target: label templates are embedded into support sentences in few-shot text classification, label channels are appended to node features in Label-GCN, and the label matrix becomes the observed variable in NMF-LAB (Burgert et al., 2024, Liu et al., 2021, Hou et al., 2024, Bellei et al., 2021, Satoh, 12 Oct 2025).

The technical motivation is also heterogeneous. In multi-label augmentation, replacement corrects additive and subtractive label noise produced by naive CutMix. In black-boxed VFL, replacement is adversarial and exploits the fact that batch-averaged gradients still encode label structure under homomorphic-encryption-based protocols. In text, graph, and factorization settings, class-based replacement is used to expose semantic or structural information that standard pipelines underuse. This suggests that class-based label replacement is best understood as a class-conditioned operation on supervision, rather than as a single canonical algorithm.

2. Label recomputation after mixed-sample augmentation

In multi-label remote sensing image classification, the central problem is that standard CutMix knows which pixels are cut and pasted, but the image-level multi-label vector does not encode where each class is located. The resulting mixed sample can therefore exhibit subtractive multi-label noise, where a class is erased from the mixed image while its label remains present, or additive multi-label noise, where a new class is pasted in while its label remains absent. The proposed remedy is a label propagation strategy that carries out a second, parallel pairing operation on pixel-level class positional information. For a cutout region R=(ra,rb,rc,rd)R=(r_a,r_b,r_c,r_d) with area AR=(rcra)(rdrb)A_R=(r_c-r_a)(r_d-r_b) and binary mask BRB_R, two images are mixed as

x~=(1B1)x1+TR2R1x(B2x2).\tilde{\mathbf{x}}=(1-B_1)\odot \mathbf{x}_1 + T^x_{R_2R_1}(B_2\odot \mathbf{x}_2).

Ordinary CutMix would use

y~=(1AR)y1+ARy2,\tilde{\mathbf{y}}=(1-A_R)\mathbf{y}_1 + A_R\mathbf{y}_2,

but this direct label mixing is unsuitable for multi-label classification. When reference maps mi[0,L]NH×W\mathbf{m}_i \in [0,L]_\mathbb{N}^{H\times W} are available, the same pairing is applied to them,

m~=(1B1)m1+TR2R1m(B2m2),y~=ϕ(m~),\tilde{\mathbf{m}}=(1-B_1)\odot \mathbf{m}_1 + T^m_{R_2R_1}(B_2\odot \mathbf{m}_2), \qquad \tilde{\mathbf{y}}=\phi(\tilde{\mathbf{m}}),

where ϕ\phi extracts the set of class labels appearing anywhere in the updated map. If no reference maps exist, class explanation masks ei{0,1}L×H×W\mathbf{e}_i \in \{0,1\}^{L\times H\times W} are used instead,

e~=(1B1)e1+TR2R1e(B2e2),y~=ψ(e~).\tilde{\mathbf{e}}=(1-B_1)\odot \mathbf{e}_1 + T^e_{R_2R_1}(B_2\odot \mathbf{e}_2), \qquad \tilde{\mathbf{y}}=\psi(\tilde{\mathbf{e}}).

The read-out AR=(rcra)(rdrb)A_R=(r_c-r_a)(r_d-r_b)0 declares class AR=(rcra)(rdrb)A_R=(r_c-r_a)(r_d-r_b)1 present if the number of activating pixels exceeds AR=(rcra)(rdrb)A_R=(r_c-r_a)(r_d-r_b)2, and heatmaps are first thresholded with AR=(rcra)(rdrb)A_R=(r_c-r_a)(r_d-r_b)3 if the explanation method does not produce binary masks. Valid CutMix boxes satisfy

AR=(rcra)(rdrb)A_R=(r_c-r_a)(r_d-r_b)4

and are sampled uniformly until AR=(rcra)(rdrb)A_R=(r_c-r_a)(r_d-r_b)5 valid rectangles are obtained (Burgert et al., 2024).

The empirical role of replacement is explicit. On DeepGlobe-ML, with box sizes in AR=(rcra)(rdrb)A_R=(r_c-r_a)(r_d-r_b)6, CutMix w/ LP reaches 82.30 mAP macro, versus 80.49 for CutMix w/o LP and 80.20 for CutOut, starting from a baseline of 78.40. Larger cut regions make LP more valuable because they increase the chance that naive CutMix would erase or add labels incorrectly. Robustness was evaluated under Mask Shift, Dilation/Erosion, Rectify Borders, Border Deformation, Segment Swap, and Class Swap; border deformations barely hurt performance, and only severe dilation/erosion or extensive segment swaps reduce the advantage more substantially. On BigEarthNet-S2, CutMix w/ LP improves over CutMix w/o LP under all box-size settings, with gains of about 2 mAP points in the best cases and best results around 76.7 mAP macro. On TreeSatAI, using Grad-CAM-derived class explanation masks with AR=(rcra)(rdrb)A_R=(r_c-r_a)(r_d-r_b)7 and AR=(rcra)(rdrb)A_R=(r_c-r_a)(r_d-r_b)8, the best setting reaches 68.91 mAP macro versus 64.67 without LP, a gain of 4.2 points (Burgert et al., 2024).

3. Target-class substitution as an attack primitive in vertical federated learning

In black-boxed vertical federated learning, class-based label replacement appears as a backdoor-style attack built on top of label inference. The setting assumes HE-protected communication: parties exchange encrypted per-sample intermediate results AR=(rcra)(rdrb)A_R=(r_c-r_a)(r_d-r_b)9, compute encrypted gradients, send them to a TTP for decryption, and only then obtain batch-averaged gradients. The passive party therefore does not see per-sample gradients directly, but still knows its own local forward and backward computations. For softmax plus cross-entropy, the gradient with respect to logits has the standard sign structure in which the true class is the unique coordinate with negative sign. The paper rewrites the batch-averaged last-layer gradient as

BRB_R0

where BRB_R1 is observed, BRB_R2 is unknown and contains the label information, and BRB_R3 is locally computable. If

BRB_R4

then the labels can be fully recovered. The practical attack uses gradient inversion by initializing BRB_R5 and minimizing

BRB_R6

over the guessed labels and latent representations (Liu et al., 2021).

Once labels have been inferred, the attacker replaces the gradient corresponding to the true label with the gradient of a target label BRB_R7: BRB_R8 Because HE is additively homomorphic, this can be done directly on encrypted values by adding or subtracting the encrypted constant BRB_R9 at selected coordinates. The paper also gives a batch/class-level replacement procedure: for each poisoned sample x~=(1B1)x1+TR2R1x(B2x2).\tilde{\mathbf{x}}=(1-B_1)\odot \mathbf{x}_1 + T^x_{R_2R_1}(B_2\odot \mathbf{x}_2).0, the passive party randomly selects a clean target sample x~=(1B1)x1+TR2R1x(B2x2).\tilde{\mathbf{x}}=(1-B_1)\odot \mathbf{x}_1 + T^x_{R_2R_1}(B_2\odot \mathbf{x}_2).1, records a pair x~=(1B1)x1+TR2R1x(B2x2).\tilde{\mathbf{x}}=(1-B_1)\odot \mathbf{x}_1 + T^x_{R_2R_1}(B_2\odot \mathbf{x}_2).2, replaces x~=(1B1)x1+TR2R1x(B2x2).\tilde{\mathbf{x}}=(1-B_1)\odot \mathbf{x}_1 + T^x_{R_2R_1}(B_2\odot \mathbf{x}_2).3 with x~=(1B1)x1+TR2R1x(B2x2).\tilde{\mathbf{x}}=(1-B_1)\odot \mathbf{x}_1 + T^x_{R_2R_1}(B_2\odot \mathbf{x}_2).4 in the forward pass, and in backpropagation replaces the encrypted gradient for sample x~=(1B1)x1+TR2R1x(B2x2).\tilde{\mathbf{x}}=(1-B_1)\odot \mathbf{x}_1 + T^x_{R_2R_1}(B_2\odot \mathbf{x}_2).5 with the gradient of sample x~=(1B1)x1+TR2R1x(B2x2).\tilde{\mathbf{x}}=(1-B_1)\odot \mathbf{x}_1 + T^x_{R_2R_1}(B_2\odot \mathbf{x}_2).6, amplified by x~=(1B1)x1+TR2R1x(B2x2).\tilde{\mathbf{x}}=(1-B_1)\odot \mathbf{x}_1 + T^x_{R_2R_1}(B_2\odot \mathbf{x}_2).7: x~=(1B1)x1+TR2R1x(B2x2).\tilde{\mathbf{x}}=(1-B_1)\odot \mathbf{x}_1 + T^x_{R_2R_1}(B_2\odot \mathbf{x}_2).8 The backdoor objective is

x~=(1B1)x1+TR2R1x(B2x2).\tilde{\mathbf{x}}=(1-B_1)\odot \mathbf{x}_1 + T^x_{R_2R_1}(B_2\odot \mathbf{x}_2).9

The reported attack accuracies for label inference are often over 90%, including 0.977 on CIFAR10 with batch size 128, 0.934 with batch size 512, 0.893 with batch size 2048, and 0.999 on CIFAR100 with batch size 128. Backdoor accuracy can exceed 90% on MNIST, NUS-WIDE, and CIFAR-style settings, and the paper uses y~=(1AR)y1+ARy2,\tilde{\mathbf{y}}=(1-A_R)\mathbf{y}_1 + A_R\mathbf{y}_2,0 as a good balance between backdoor success and utility. The proposed defense, CoAE, maps y~=(1AR)y1+ARy2,\tilde{\mathbf{y}}=(1-A_R)\mathbf{y}_1 + A_R\mathbf{y}_2,1, y~=(1AR)y1+ARy2,\tilde{\mathbf{y}}=(1-A_R)\mathbf{y}_1 + A_R\mathbf{y}_2,2 and optimizes

y~=(1AR)y1+ARy2,\tilde{\mathbf{y}}=(1-A_R)\mathbf{y}_1 + A_R\mathbf{y}_2,3

It is reported to outperform DP, gradient sparsification, and MARVELL in the label inference setting, and the paper notes that without entropy loss, y~=(1AR)y1+ARy2,\tilde{\mathbf{y}}=(1-A_R)\mathbf{y}_1 + A_R\mathbf{y}_2,4, the CoAE becomes much less robust because it merely permutes labels and does not sufficiently confuse the attacker (Liu et al., 2021).

4. Label semantics as an input-side replacement signal in few-shot text classification

In few-shot text classification, class-based label replacement takes the form of replacing the usual label-agnostic support encoding with label-templated inputs. The method argues that standard prototypical approaches underuse class labels because labels are often only used at the final classification step, while simple insertion of labels into text can make the input unnatural. The proposed solution is a manually designed template, with the best-performing choice reported as “Overall, the topic of the text is”. For class y~=(1AR)y1+ARy2,\tilde{\mathbf{y}}=(1-A_R)\mathbf{y}_1 + A_R\mathbf{y}_2,5, the label text is written as

y~=(1AR)y1+ARy2,\tilde{\mathbf{y}}=(1-A_R)\mathbf{y}_1 + A_R\mathbf{y}_2,6

and BERT encodes the pair

y~=(1AR)y1+ARy2,\tilde{\mathbf{y}}=(1-A_R)\mathbf{y}_1 + A_R\mathbf{y}_2,7

Support and query samples are treated differently: support samples concatenate the template with the original sentence and the class label information, whereas query samples do not have their true labels available and are kept in the original form. The episode is represented as

y~=(1AR)y1+ARy2,\tilde{\mathbf{y}}=(1-A_R)\mathbf{y}_1 + A_R\mathbf{y}_2,8

The templated encoding is coupled with supervised contrastive learning over the merged episode

y~=(1AR)y1+ARy2,\tilde{\mathbf{y}}=(1-A_R)\mathbf{y}_1 + A_R\mathbf{y}_2,9

where positives for mi[0,L]NH×W\mathbf{m}_i \in [0,L]_\mathbb{N}^{H\times W}0 are

mi[0,L]NH×W\mathbf{m}_i \in [0,L]_\mathbb{N}^{H\times W}1

and cosine similarity is used: mi[0,L]NH×W\mathbf{m}_i \in [0,L]_\mathbb{N}^{H\times W}2 The per-sample contrastive loss is

mi[0,L]NH×W\mathbf{m}_i \in [0,L]_\mathbb{N}^{H\times W}3

Prototype construction replaces averaging with an attention mechanism: mi[0,L]NH×W\mathbf{m}_i \in [0,L]_\mathbb{N}^{H\times W}4 and the class prototype becomes

mi[0,L]NH×W\mathbf{m}_i \in [0,L]_\mathbb{N}^{H\times W}5

Query classification uses

mi[0,L]NH×W\mathbf{m}_i \in [0,L]_\mathbb{N}^{H\times W}6

with prototype-network loss

mi[0,L]NH×W\mathbf{m}_i \in [0,L]_\mathbb{N}^{H\times W}7

and joint optimization

mi[0,L]NH×W\mathbf{m}_i \in [0,L]_\mathbb{N}^{H\times W}8

where mi[0,L]NH×W\mathbf{m}_i \in [0,L]_\mathbb{N}^{H\times W}9 increases linearly from 0 to 1 during training.

The reported results indicate that label semantics contribute the largest single improvement. On 20News, Amazon, HuffPost, and Reuters, the method outperforms nearly all baselines in both 1-shot and 5-shot settings. Average improvements over the best baseline are +4.7% accuracy and +5.4% F1 in 1-shot, and +2.6% accuracy and +2.9% F1 in 5-shot. On HuffPost, the 1-shot gains are +7.5% accuracy and +8.4% F1. In ablation, label template alone gives +7.22% accuracy in 1-shot and +3.37% in 5-shot; for F1, label template alone gives +8.12% in 1-shot and +3.00% in 5-shot. The full model gives +8.12% accuracy in 1-shot and +3.48% in 5-shot, and +9.26% F1 in 1-shot and +3.15% in 5-shot. Faster convergence than ContrastNet and t-SNE plots with more compact intra-class clusters and clearer inter-class boundaries are also reported (Hou et al., 2024).

5. Structured label propagation and reconstruction in graphs, factorization, and zero-shot learning

A graph-based form of class-based label replacement is implemented by Label-GCN. The method appends one-hot labels to node features so that the input dimension becomes m~=(1B1)m1+TR2R1m(B2m2),y~=ϕ(m~),\tilde{\mathbf{m}}=(1-B_1)\odot \mathbf{m}_1 + T^m_{R_2R_1}(B_2\odot \mathbf{m}_2), \qquad \tilde{\mathbf{y}}=\phi(\tilde{\mathbf{m}}),0, but modifies only the first layer so that self-loops are removed selectively for the label coordinates: m~=(1B1)m1+TR2R1m(B2m2),y~=ϕ(m~),\tilde{\mathbf{m}}=(1-B_1)\odot \mathbf{m}_1 + T^m_{R_2R_1}(B_2\odot \mathbf{m}_2), \qquad \tilde{\mathbf{y}}=\phi(\tilde{\mathbf{m}}),1 while deeper layers remain standard GCN layers,

m~=(1B1)m1+TR2R1m(B2m2),y~=ϕ(m~),\tilde{\mathbf{m}}=(1-B_1)\odot \mathbf{m}_1 + T^m_{R_2R_1}(B_2\odot \mathbf{m}_2), \qquad \tilde{\mathbf{y}}=\phi(\tilde{\mathbf{m}}),2

This masks the label of the center node while still allowing labels of neighboring nodes to flow in. The method supports binary and multi-class settings, works in both transductive and inductive regimes, and improves as more support labels are available at inference time. On the Elliptic dataset, it improves illicit-class detection and is reported to be more robust to the “dark market shutdown,” while the paper also notes that illicit nodes are only about 2% of the graph (Bellei et al., 2021).

A matrix-factorization form of replacement is given by NMF-LAB, which inverts the usual roles of features and labels by treating the label matrix as the observation: m~=(1B1)m1+TR2R1m(B2m2),y~=ϕ(m~),\tilde{\mathbf{m}}=(1-B_1)\odot \mathbf{m}_1 + T^m_{R_2R_1}(B_2\odot \mathbf{m}_2), \qquad \tilde{\mathbf{y}}=\phi(\tilde{\mathbf{m}}),3 Here m~=(1B1)m1+TR2R1m(B2m2),y~=ϕ(m~),\tilde{\mathbf{m}}=(1-B_1)\odot \mathbf{m}_1 + T^m_{R_2R_1}(B_2\odot \mathbf{m}_2), \qquad \tilde{\mathbf{y}}=\phi(\tilde{\mathbf{m}}),4 is the label matrix, m~=(1B1)m1+TR2R1m(B2m2),y~=ϕ(m~),\tilde{\mathbf{m}}=(1-B_1)\odot \mathbf{m}_1 + T^m_{R_2R_1}(B_2\odot \mathbf{m}_2), \qquad \tilde{\mathbf{y}}=\phi(\tilde{\mathbf{m}}),5 is the covariate matrix, m~=(1B1)m1+TR2R1m(B2m2),y~=ϕ(m~),\tilde{\mathbf{m}}=(1-B_1)\odot \mathbf{m}_1 + T^m_{R_2R_1}(B_2\odot \mathbf{m}_2), \qquad \tilde{\mathbf{y}}=\phi(\tilde{\mathbf{m}}),6 is a basis matrix, and m~=(1B1)m1+TR2R1m(B2m2),y~=ϕ(m~),\tilde{\mathbf{m}}=(1-B_1)\odot \mathbf{m}_1 + T^m_{R_2R_1}(B_2\odot \mathbf{m}_2), \qquad \tilde{\mathbf{y}}=\phi(\tilde{\mathbf{m}}),7 maps covariates to latent coefficients. With m~=(1B1)m1+TR2R1m(B2m2),y~=ϕ(m~),\tilde{\mathbf{m}}=(1-B_1)\odot \mathbf{m}_1 + T^m_{R_2R_1}(B_2\odot \mathbf{m}_2), \qquad \tilde{\mathbf{y}}=\phi(\tilde{\mathbf{m}}),8, coefficients are normalized by

m~=(1B1)m1+TR2R1m(B2m2),y~=ϕ(m~),\tilde{\mathbf{m}}=(1-B_1)\odot \mathbf{m}_1 + T^m_{R_2R_1}(B_2\odot \mathbf{m}_2), \qquad \tilde{\mathbf{y}}=\phi(\tilde{\mathbf{m}}),9

and predicted class probabilities are reconstructed as

ϕ\phi0

Columns of ϕ\phi1 are normalized to sum to one, often with ϕ\phi2, so bases often align closely with classes and can become close to a one-hot basis matrix up to permutation. Unlabeled examples can be encoded as uniform distributions, for example ϕ\phi3, which supports semi-supervised learning. The paper reports that NMF-LAB (direct) was often weaker than nonlinear baselines, while NMF-LAB (kernel) was much stronger: on Iris, direct 66.7% and kernel 95.5%; on Penguins, direct 79.1% and kernel 97.9%; on Digits, direct 68.0% and kernel 94.6%. On MNIST, direct NMF-LAB reaches 69.1%, kernel500 91.1%, kernel1000 93.7%, and full kernel 96.0% (Satoh, 12 Oct 2025).

A zero-shot form of replacement is implemented by the Class Label Autoencoder, which uses the class label space as the intermediate latent space. The encoder maps visual features to labels by

ϕ\phi4

and the decoder reconstructs features with tied weights,

ϕ\phi5

The optimization objective is

ϕ\phi6

For seen and unseen classes, the method uses

ϕ\phi7

and unseen label scores are computed by

ϕ\phi8

Prediction is then

ϕ\phi9

CLA fuses multiple semantic embeddings through linear combinations of ei{0,1}L×H×W\mathbf{e}_i \in \{0,1\}^{L\times H\times W}0, ei{0,1}L×H×W\mathbf{e}_i \in \{0,1\}^{L\times H\times W}1, and ei{0,1}L×H×W\mathbf{e}_i \in \{0,1\}^{L\times H\times W}2, and iteratively refines unseen structure. Reported gains over SAE and SP are at least +2% on AwA, +3.7% on CUB, +3.2% on Dogs, and +0.1% on ImNet-2. In multi-semantic settings, examples include 88.8 on AwA with all semantics, 60.3 on CUB with all semantics, and 48.6 on Dogs without attributes; on ImNet-2, Top-1 through Top-5 are 13.6, 19.6, 23.3, 26.0, and 28.3 (Lin et al., 2018).

6. Weak supervision, noisy labels, and the limits of literal replacement

Not every class-based modification of supervision is a literal label replacement. In scribble-based weakly supervised semantic segmentation, Class-driven Scribble Promotion begins from sparse scribbles ei{0,1}L×H×W\mathbf{e}_i \in \{0,1\}^{L\times H\times W}3, extracts image-level class labels ei{0,1}L×H×W\mathbf{e}_i \in \{0,1\}^{L\times H\times W}4, trains a classification model, and generates dense pseudo-labels

ei{0,1}L×H×W\mathbf{e}_i \in \{0,1\}^{L\times H\times W}5

With CAMs, class ei{0,1}L×H×W\mathbf{e}_i \in \{0,1\}^{L\times H\times W}6 is represented by

ei{0,1}L×H×W\mathbf{e}_i \in \{0,1\}^{L\times H\times W}7

The segmentation loss combines scribble supervision and pseudo-label supervision,

ei{0,1}L×H×W\mathbf{e}_i \in \{0,1\}^{L\times H\times W}8

with a partial cross-entropy on scribbled pixels and a smoothed cross-entropy on the pseudo-mask using ei{0,1}L×H×W\mathbf{e}_i \in \{0,1\}^{L\times H\times W}9. To correct noisy pseudo-label guidance, the method adds a Localization Rectification Module with

e~=(1B1)e1+TR2R1e(B2e2),y~=ψ(e~).\tilde{\mathbf{e}}=(1-B_1)\odot \mathbf{e}_1 + T^e_{R_2R_1}(B_2\odot \mathbf{e}_2), \qquad \tilde{\mathbf{y}}=\psi(\tilde{\mathbf{e}}).0

and a Distance Entropy Loss

e~=(1B1)e1+TR2R1e(B2e2),y~=ψ(e~).\tilde{\mathbf{e}}=(1-B_1)\odot \mathbf{e}_1 + T^e_{R_2R_1}(B_2\odot \mathbf{e}_2), \qquad \tilde{\mathbf{y}}=\psi(\tilde{\mathbf{e}}).1

leading to

e~=(1B1)e1+TR2R1e(B2e2),y~=ψ(e~).\tilde{\mathbf{e}}=(1-B_1)\odot \mathbf{e}_1 + T^e_{R_2R_1}(B_2\odot \mathbf{e}_2), \qquad \tilde{\mathbf{y}}=\psi(\tilde{\mathbf{e}}).2

The paper explicitly states that this is not a literal pixel-level replacement of scribble labels with class labels but a hybrid scribble-plus-class-guided label promotion method. On ScribbleSup, baseline scribble-only with ResNet101 + DeepLabV3+ gives 66.2 mIoU val and 69.7 test, while the proposed method gives 75.9 mIoU val and 76.0 test. Using only scribble or only pseudo-label gives around 67% mIoU, but using both gives 72.13%; LoRM improves performance from 67.33% to 73.64%; and under scribble-shrink or scribble-drop, methods like AGMM and TEL lose about 10% when scribbles shrink to points, while this method drops by only about 1% (Zhang et al., 2024).

A related but distinct case is class-aware label cleaning under noisy supervision. CPC starts from a noisy-label dataset e~=(1B1)e1+TR2R1e(B2e2),y~=ψ(e~).\tilde{\mathbf{e}}=(1-B_1)\odot \mathbf{e}_1 + T^e_{R_2R_1}(B_2\odot \mathbf{e}_2), \qquad \tilde{\mathbf{y}}=\psi(\tilde{\mathbf{e}}).3, uses a global two-component GMM over per-sample cross-entropy losses to obtain a first-pass partition e~=(1B1)e1+TR2R1e(B2e2),y~=ψ(e~).\tilde{\mathbf{e}}=(1-B_1)\odot \mathbf{e}_1 + T^e_{R_2R_1}(B_2\odot \mathbf{e}_2), \qquad \tilde{\mathbf{y}}=\psi(\tilde{\mathbf{e}}).4 and e~=(1B1)e1+TR2R1e(B2e2),y~=ψ(e~).\tilde{\mathbf{e}}=(1-B_1)\odot \mathbf{e}_1 + T^e_{R_2R_1}(B_2\odot \mathbf{e}_2), \qquad \tilde{\mathbf{y}}=\psi(\tilde{\mathbf{e}}).5, and then introduces class prototypes e~=(1B1)e1+TR2R1e(B2e2),y~=ψ(e~).\tilde{\mathbf{e}}=(1-B_1)\odot \mathbf{e}_1 + T^e_{R_2R_1}(B_2\odot \mathbf{e}_2), \qquad \tilde{\mathbf{y}}=\psi(\tilde{\mathbf{e}}).6. The class-aware clean score is

e~=(1B1)e1+TR2R1e(B2e2),y~=ψ(e~).\tilde{\mathbf{e}}=(1-B_1)\odot \mathbf{e}_1 + T^e_{R_2R_1}(B_2\odot \mathbf{e}_2), \qquad \tilde{\mathbf{y}}=\psi(\tilde{\mathbf{e}}).7

and the clean/noisy decision rule is

e~=(1B1)e1+TR2R1e(B2e2),y~=ψ(e~).\tilde{\mathbf{e}}=(1-B_1)\odot \mathbf{e}_1 + T^e_{R_2R_1}(B_2\odot \mathbf{e}_2), \qquad \tilde{\mathbf{y}}=\psi(\tilde{\mathbf{e}}).8

Prototype learning uses clean samples from e~=(1B1)e1+TR2R1e(B2e2),y~=ψ(e~).\tilde{\mathbf{e}}=(1-B_1)\odot \mathbf{e}_1 + T^e_{R_2R_1}(B_2\odot \mathbf{e}_2), \qquad \tilde{\mathbf{y}}=\psi(\tilde{\mathbf{e}}).9, noisy samples from AR=(rcra)(rdrb)A_R=(r_c-r_a)(r_d-r_b)00, and high-confidence pseudo-clean samples AR=(rcra)(rdrb)A_R=(r_c-r_a)(r_d-r_b)01, with full objective

AR=(rcra)(rdrb)A_R=(r_c-r_a)(r_d-r_b)02

The method is justified through an EM-style interpretation in which the exact E-step posterior AR=(rcra)(rdrb)A_R=(r_c-r_a)(r_d-r_b)03 is approximated by the GMM posterior AR=(rcra)(rdrb)A_R=(r_c-r_a)(r_d-r_b)04. CPC is therefore a class-aware cleaner for partitioning and relabeling confidence, rather than a direct target-label substitution. The paper reports that CPC outperforms both class-agnostic GMM cleaner and naive class-aware GMM cleaner in cleaner AUC, improves final accuracy on CIFAR-10, CIFAR-100, Clothing1M, and WebVision, and that using GMM to warm up prototype learning yields better prototypes than prototype learning without GMM (Huang et al., 2022).

The common misconception is that every use of class information to modify supervision is a literal replacement of one label by another. The literature instead spans exact recomputation from propagated class-position evidence, adversarial target-class substitution, input-side semantic augmentation, neighborhood label propagation, latent probabilistic label reconstruction, class-informed pseudo-label generation, and class-aware clean/noisy repartitioning. A plausible implication is that the unifying property is not the data structure being modified, but the decision to let class-conditioned structure overwrite or refine an original supervisory signal when the original signal is incomplete, noisy, strategically manipulated, or semantically underexploited.

Topic to Video (Beta)

No one has generated a video about this topic yet.

Whiteboard

No one has generated a whiteboard explanation for this topic yet.

Follow Topic

Get notified by email when new papers are published related to Class-based Label Replacement.