Class-Unbiased Model (Cls-unbias)

• Class-Unbiased Models are frameworks that ensure each class contributes equally to risk estimation, maintaining statistical parity in weak supervision, diagnosis, and incremental learning.
• They employ techniques such as risk rewriting, loss equalization, and shared attention steering to mitigate class imbalance and prevent reliance on spurious features.
• Empirical studies show that these methods improve generalization and stability across tasks, from unbiased risk estimation in aggregate data to robust few-shot learning.

A Class-Unbiased Model (Cls-unbias) denotes a model or training principle designed to avoid systematic class-wise distortion, but the term is used in more than one technical sense in recent literature. In classification from aggregate observations, class-unbiasedness refers to a strict statistical property: the learning objective defined on aggregate labels is exactly equal to the ordinary instance-level classification risk, so no class is systematically over- or under-weighted by the weak-supervision mechanism (Wei et al., 2023). In medical diagnosis, Cls-unbias refers to a supervised objective that equalizes class-wise losses and combines this with class-wise group distributionally robust optimization to mitigate both class imbalance and class-feature bias (Zuo et al., 9 Aug 2025). In few-shot class-incremental learning, closely related usage appears in CASP, where class-shared trainable bias parameters are injected into the query, key, and value projections of the CLS token so that the same attention steering is applied to base and novel classes (Huang et al., 23 Jan 2026). A broader surrounding literature on unbiased visual representations and supervised contrastive debiasing studies bias-robust classifiers whose predictions depend less on spurious attributes and remain stable under bias shifts (Barbano et al., 2022, Barbano et al., 2022).

1. Terminological scope and core ideas

The phrase class-unbiased does not designate a single universal construction. Instead, recent work uses it to denote distinct but related goals: preserving class-wise contribution to risk under weak supervision, preventing class-specific reliance on spurious features, or enforcing class-agnostic representation updates in continual learning.

In the CFAO formulation, unbiasedness is defined through risk rewriting. A method is called unbiased if the risk it defines and estimates from aggregate data is exactly equal to the ordinary classification risk

$$R(f) = \mathbb{E}_{p(\boldsymbol{x},y)}\big[\mathcal{L}(\boldsymbol{x}, y; f)\big].$$

Because the rewriting is symmetric across classes and compatible with arbitrary losses, the resulting estimator is also described as class-unbiased: every class contributes to the expected risk exactly as it would under direct supervision (Wei et al., 2023).

In medical diagnosis, the central concern is class-feature bias, defined as reliance on features that are correlated with only a subset of classes, leading to biased performance and poor generalization on other classes. The proposed Cls-unbias model is characterized through class-wise loss equalization and a class-wise group DRO objective, with the explicit aim of mitigating both class imbalance and class-feature bias simultaneously (Zuo et al., 9 Aug 2025).

In FSCIL, CASP does not formulate class-unbiasedness as risk equality or class-wise loss equality. Instead, it introduces class-shared trainable bias parameters into the CLS token attention mechanism. Because these parameters are shared across all classes, optimized only once in the base session, and then frozen, the attention steering is described as class-unbiased in the sense that it is identical and agnostic with respect to which class is being processed (Huang et al., 23 Jan 2026).

A plausible implication is that Cls-unbias should be understood as a family of design principles centered on one invariant requirement: the learning mechanism should not systematically privilege some classes through the supervision protocol, the feature selection process, or the adaptation rule.

2. Risk-unbiased and class-unbiased learning from aggregate observations

The CFAO setting replaces instance labels with group-level aggregate labels. One observes groups of instances $$\boldsymbol{x}_{1:m} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_m)$$ and an aggregate label $z = g(y_{1:m})$, while the individual labels $y_{1:m}$ remain latent. This unified framework includes multiple-instance learning, learning from label proportions, classification from pairwise similarity, triplet comparison, ordinal ranks, and ordinal triplets (Wei et al., 2023).

The key result is an unbiased aggregate-risk form:

$$R(f) \;=\; \mathbb{E}_{p(\boldsymbol{x}_{1:m}, z)}\big[\mathcal{L}_{\mathrm{agg}}(\boldsymbol{x}_{1:m}, z; f)\big],$$

with aggregate loss

$$\mathcal{L}_{\mathrm{agg}}(\boldsymbol{x}_{1:m}, z; f) =\frac{1}{m}\,\frac{1}{p(z\mid\boldsymbol{x}_{1:m})} \sum_{i=1}^m \sum_{j=1}^k p(z, y_i=j \mid \boldsymbol{x}_{1:m})\,\mathcal{L}(\boldsymbol{x}_i, j; f).$$

Equivalently,

$$\mathcal{L}_{\mathrm{agg}}(\boldsymbol{x}_{1:m}, z; f) = \frac{1}{m} \sum_{i=1}^m \sum_{j=1}^k p(y_i=j\mid\boldsymbol{x}_{1:m}, z)\,\mathcal{L}(\boldsymbol{x}_i, j; f).$$

Thus each possible label for each instance is weighted by its posterior probability given the group and the aggregate label. The method “weighs the importance of each label for each instance in the group,” providing purified supervision while preserving the original class-wise contribution to risk (Wei et al., 2023).

This formulation is universal in three explicit senses. First, it is compatible with arbitrary classification losses, including cross-entropy, logistic, and hinge loss. Second, it applies to arbitrary group sizes. Third, it applies to arbitrary aggregate functions $g$, provided the assumptions hold. The resulting empirical estimator

$$\widehat{R}_{\mathrm{agg}}(f) = \frac{1}{n}\sum_{i=1}^n \mathcal{L}_{\mathrm{agg}}(\boldsymbol{x}^{(i)}_{1:m}, z^{(i)}; f)$$

is an unbiased estimator of $R(f)$, and the paper further states risk consistency under standard conditions.

The derivation relies on two assumptions:

$$\boldsymbol{x}_{1:m} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_m)$$0

and

$$\boldsymbol{x}_{1:m} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_m)$$1

Under these assumptions, the joint factorization

$$\boldsymbol{x}_{1:m} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_m)$$2

enables the exact rewriting from instance-level risk to group-level expectation.

This strict statistical notion of class-unbiasedness should not be conflated with demographic fairness. The paper explicitly frames it as an unbiased estimator of the true classification risk rather than a fairness criterion. Its significance lies in eliminating the mismatch between optimizing a group-level objective and the actual instance-level classification objective (Wei et al., 2023).

3. Cls-unbias as class-wise loss equalization in medical diagnosis

The medical diagnosis formulation begins from class-feature bias. Let $$\boldsymbol{x}_{1:m} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_m)$$3 with labels $$\boldsymbol{x}_{1:m} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_m)$$4. For subsets of classes $$\boldsymbol{x}_{1:m} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_m)$$5, the corresponding subsets $$\boldsymbol{x}_{1:m} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_m)$$6 partition the dataset. The paper defines class-feature bias through conditions such as

$$\boldsymbol{x}_{1:m} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_m)$$7

for some subset of classes, and

$$\boldsymbol{x}_{1:m} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_m)$$8

for at least one other subset. In that case, $$\boldsymbol{x}_{1:m} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_m)$$9 is informative for some classes but not others, so it can cause class-feature bias (Zuo et al., 9 Aug 2025).

The proposed Cls-unbias model seeks features that are equally informative for positive and negative classes. This is expressed through the condition

$z = g(y_{1:m})$0

Using the relation

$z = g(y_{1:m})$1

the paper argues that when the model posterior approximates the true posterior well, this reduces approximately to

$z = g(y_{1:m})$2

Accordingly, a Cls-unbias model is informally defined as one whose class-wise classification losses are equalized, or nearly equalized, so that the features it uses are equally informative for all classes (Zuo et al., 9 Aug 2025).

The first component is the class-wise inequality loss

$z = g(y_{1:m})$3

The paper emphasizes a two-sided effect: minimizing this loss decreases the larger class-wise loss and increases the smaller one. The latter is deliberate. It penalizes over-specialization to one class and pushes the model away from features that are only predictive for that class.

The second component is the class-wise group DRO objective

$z = g(y_{1:m})$4

Here, higher-loss classes receive larger weights, with $z = g(y_{1:m})$5 controlling how sharply the worst-loss class is emphasized and $z = g(y_{1:m})$6 denoting stop-gradient. When the class-wise losses become equal, the objective simplifies to a per-class average. The combined training objective is

$z = g(y_{1:m})$7

The paper further specifies a dynamic schedule: $z = g(y_{1:m})$8 is linearly increased from $z = g(y_{1:m})$9 to $y_{1:m}$0, while $y_{1:m}$1 is linearly decreased from $y_{1:m}$2 to $y_{1:m}$3. This lets the model first obtain better class-wise estimates through DRO and then progressively enforce stronger loss equalization (Zuo et al., 9 Aug 2025).

4. Class-shared CLS attention steering in few-shot class-incremental learning

CASP situates class-unbiasedness in the context of FSCIL, where a base session $y_{1:m}$4 is followed by incremental sessions $y_{1:m}$5, each an $y_{1:m}$6-way $y_{1:m}$7-shot task, with no access to past data during later sessions. The problem is to classify over all classes seen so far without catastrophic forgetting and without overfitting to the few samples of novel classes (Huang et al., 23 Jan 2026).

The method is motivated by the role of the CLS token in Vision Transformers. In prototypical FSCIL systems, class prototypes are computed as

$y_{1:m}$8

typically using the CLS embedding. CASP therefore modifies the attention mechanism associated with the CLS token rather than introducing class-specific prompt pools.

For the CLS token in each MSA layer,

$y_{1:m}$9

CASP introduces learnable additive vectors $$R(f) \;=\; \mathbb{E}_{p(\boldsymbol{x}_{1:m}, z)}\big[\mathcal{L}_{\mathrm{agg}}(\boldsymbol{x}_{1:m}, z; f)\big],$$0:

$$R(f) \;=\; \mathbb{E}_{p(\boldsymbol{x}_{1:m}, z)}\big[\mathcal{L}_{\mathrm{agg}}(\boldsymbol{x}_{1:m}, z; f)\big],$$1

These are class-shared: the same vectors are used for all images and all classes, applied only to the CLS token, and embedded into all layers of the backbone. A separate CLS Domain Adaptation Prompt adds a single vector to the initial CLS token,

$$R(f) \;=\; \mathbb{E}_{p(\boldsymbol{x}_{1:m}, z)}\big[\mathcal{L}_{\mathrm{agg}}(\boldsymbol{x}_{1:m}, z; f)\big],$$2

During training, Perturbed CAGP applies dropout to the prompt vectors:

$$R(f) \;=\; \mathbb{E}_{p(\boldsymbol{x}_{1:m}, z)}\big[\mathcal{L}_{\mathrm{agg}}(\boldsymbol{x}_{1:m}, z; f)\big],$$3

The paper characterizes the class-unbiased behavior of CASP through parameter sharing and freezing. The prompts are not conditioned on class labels, not retrieved from a class-indexed pool, and not changed across sessions. They are optimized only once in the base session and then frozen. Consequently, the same attention bias is applied to base and novel classes, and no class index or task index influences which prompts are applied (Huang et al., 23 Jan 2026).

CASP further includes Manifold Token Mixup in shallow feature space:

$$R(f) \;=\; \mathbb{E}_{p(\boldsymbol{x}_{1:m}, z)}\big[\mathcal{L}_{\mathrm{agg}}(\boldsymbol{x}_{1:m}, z; f)\big],$$4

with total loss

$$R(f) \;=\; \mathbb{E}_{p(\boldsymbol{x}_{1:m}, z)}\big[\mathcal{L}_{\mathrm{agg}}(\boldsymbol{x}_{1:m}, z; f)\big],$$5

The stated purpose is to synthesize potential new class features, improve generalization, and reserve representation capacity for upcoming tasks. Training is restricted to the base session; the backbone including CASP prompts and CDAP is frozen afterward, and incremental learning is prototype-based.

This usage of class-unbiasedness differs from the medical and CFAO formulations. It is neither a risk identity nor a class-wise loss-equality criterion. Instead, it is a class-agnostic representational intervention: the same CLS steering rule is reused for all classes and all future sessions (Huang et al., 23 Jan 2026).

5. Relation to unbiased representation learning and supervised contrastive debiasing

A broader literature studies bias-robust classifiers whose predictions are less dependent on spurious attributes, even when the term Cls-unbias is not always used explicitly. In “Unsupervised Learning of Unbiased Visual Representations,” the objective is to learn a representation $$R(f) \;=\; \mathbb{E}_{p(\boldsymbol{x}_{1:m}, z)}\big[\mathcal{L}_{\mathrm{agg}}(\boldsymbol{x}_{1:m}, z; f)\big],$$6 that is informative about the target while being as uninformative as possible about the bias. The proposed U-EnD pipeline first trains a standard bias-capturing model, then extracts pseudo-bias labels by PCA and KMeans clustering, and finally trains an unbiased classifier with EnD regularization using the pseudo-bias labels (Barbano et al., 2022).

The EnD regularizer combines a disentangling term and an entangling term. For normalized representations $$R(f) \;=\; \mathbb{E}_{p(\boldsymbol{x}_{1:m}, z)}\big[\mathcal{L}_{\mathrm{agg}}(\boldsymbol{x}_{1:m}, z; f)\big],$$7, with bias-same set $$R(f) \;=\; \mathbb{E}_{p(\boldsymbol{x}_{1:m}, z)}\big[\mathcal{L}_{\mathrm{agg}}(\boldsymbol{x}_{1:m}, z; f)\big],$$8 and target-same but bias-different set $$R(f) \;=\; \mathbb{E}_{p(\boldsymbol{x}_{1:m}, z)}\big[\mathcal{L}_{\mathrm{agg}}(\boldsymbol{x}_{1:m}, z; f)\big],$$9, the per-sample regularizer is

$$\mathcal{L}_{\mathrm{agg}}(\boldsymbol{x}_{1:m}, z; f) =\frac{1}{m}\,\frac{1}{p(z\mid\boldsymbol{x}_{1:m})} \sum_{i=1}^m \sum_{j=1}^k p(z, y_i=j \mid \boldsymbol{x}_{1:m})\,\mathcal{L}(\boldsymbol{x}_i, j; f).$$0

Minimizing this loss decorrelates representations that share the same bias while aligning representations with the same target across different biases. The paper positions the resulting classifier as robust to bias shifts and more stable across bias conditions (Barbano et al., 2022).

A complementary line appears in “Unbiased Supervised Contrastive Learning,” which analyzes why InfoNCE and SupCon can fail under biased data and proposes $$\mathcal{L}_{\mathrm{agg}}(\boldsymbol{x}_{1:m}, z; f) =\frac{1}{m}\,\frac{1}{p(z\mid\boldsymbol{x}_{1:m})} \sum_{i=1}^m \sum_{j=1}^k p(z, y_i=j \mid \boldsymbol{x}_{1:m})\,\mathcal{L}(\boldsymbol{x}_i, j; f).$$1-SupInfoNCE together with FairKL (Barbano et al., 2022). For an anchor with positives $$\mathcal{L}_{\mathrm{agg}}(\boldsymbol{x}_{1:m}, z; f) =\frac{1}{m}\,\frac{1}{p(z\mid\boldsymbol{x}_{1:m})} \sum_{i=1}^m \sum_{j=1}^k p(z, y_i=j \mid \boldsymbol{x}_{1:m})\,\mathcal{L}(\boldsymbol{x}_i, j; f).$$2 and negatives $$\mathcal{L}_{\mathrm{agg}}(\boldsymbol{x}_{1:m}, z; f) =\frac{1}{m}\,\frac{1}{p(z\mid\boldsymbol{x}_{1:m})} \sum_{i=1}^m \sum_{j=1}^k p(z, y_i=j \mid \boldsymbol{x}_{1:m})\,\mathcal{L}(\boldsymbol{x}_i, j; f).$$3, the loss

$$\mathcal{L}_{\mathrm{agg}}(\boldsymbol{x}_{1:m}, z; f) =\frac{1}{m}\,\frac{1}{p(z\mid\boldsymbol{x}_{1:m})} \sum_{i=1}^m \sum_{j=1}^k p(z, y_i=j \mid \boldsymbol{x}_{1:m})\,\mathcal{L}(\boldsymbol{x}_i, j; f).$$4

enforces a margin condition between positives and negatives. FairKL then regularizes the distance geometry so that bias-aligned and bias-conflicting samples have matched distance distributions. The stated goal is an unbiased representation in which predictions do not systematically depend on bias attributes (Barbano et al., 2022).

These representation-centric approaches operate at a different level from the CFAO and medical formulations. They do not define class-unbiasedness through exact risk preservation or explicit class-wise loss equality. Instead, they aim to suppress bias structure in latent space so that downstream class predictions remain stable when spurious correlations change. This suggests a broader organizing distinction between objective-level class-unbiasedness and representation-level class-unbiasedness.

6. Empirical evidence, assumptions, and limitations

The empirical profile of class-unbiased methods varies with the formulation. In CFAO, the universal unbiased method is evaluated on pairwise similarity, triplet comparison, LLP, and MIL. The paper reports that the unbiased method consistently outperforms baselines on almost all datasets in pairwise similarity, is best or tied best on almost all datasets for triplet comparison, matches or surpasses strong methods in LLP, and clearly dominates baselines in MIL, while also remaining better under reduced sample size on CIFAR-10 (Wei et al., 2023).

In medical diagnosis, Cls-unbias is evaluated on DAIC-WOZ, MODMA, ADReSS, BreastMNIST, and RetinaMNIST. The paper reports MF1 improvements over ERM and class-weighted ERM on all five datasets, including a gain from $$\mathcal{L}_{\mathrm{agg}}(\boldsymbol{x}_{1:m}, z; f) =\frac{1}{m}\,\frac{1}{p(z\mid\boldsymbol{x}_{1:m})} \sum_{i=1}^m \sum_{j=1}^k p(z, y_i=j \mid \boldsymbol{x}_{1:m})\,\mathcal{L}(\boldsymbol{x}_i, j; f).$$5 to $$\mathcal{L}_{\mathrm{agg}}(\boldsymbol{x}_{1:m}, z; f) =\frac{1}{m}\,\frac{1}{p(z\mid\boldsymbol{x}_{1:m})} \sum_{i=1}^m \sum_{j=1}^k p(z, y_i=j \mid \boldsymbol{x}_{1:m})\,\mathcal{L}(\boldsymbol{x}_i, j; f).$$6 on DAIC-WOZ and from $$\mathcal{L}_{\mathrm{agg}}(\boldsymbol{x}_{1:m}, z; f) =\frac{1}{m}\,\frac{1}{p(z\mid\boldsymbol{x}_{1:m})} \sum_{i=1}^m \sum_{j=1}^k p(z, y_i=j \mid \boldsymbol{x}_{1:m})\,\mathcal{L}(\boldsymbol{x}_i, j; f).$$7 to $$\mathcal{L}_{\mathrm{agg}}(\boldsymbol{x}_{1:m}, z; f) =\frac{1}{m}\,\frac{1}{p(z\mid\boldsymbol{x}_{1:m})} \sum_{i=1}^m \sum_{j=1}^k p(z, y_i=j \mid \boldsymbol{x}_{1:m})\,\mathcal{L}(\boldsymbol{x}_i, j; f).$$8 on RetinaMNIST. On synthetic data, Cls-unbias reduces reliance on the spurious feature $$\mathcal{L}_{\mathrm{agg}}(\boldsymbol{x}_{1:m}, z; f) =\frac{1}{m}\,\frac{1}{p(z\mid\boldsymbol{x}_{1:m})} \sum_{i=1}^m \sum_{j=1}^k p(z, y_i=j \mid \boldsymbol{x}_{1:m})\,\mathcal{L}(\boldsymbol{x}_i, j; f).$$9 and yields a decision boundary closer to the ideal class-unbiased vertical boundary (Zuo et al., 9 Aug 2025).

In FSCIL, CASP reports strong average and last-session accuracies on CUB200, CIFAR100, and ImageNet-R, while using only $$\mathcal{L}_{\mathrm{agg}}(\boldsymbol{x}_{1:m}, z; f) = \frac{1}{m} \sum_{i=1}^m \sum_{j=1}^k p(y_i=j\mid\boldsymbol{x}_{1:m}, z)\,\mathcal{L}(\boldsymbol{x}_i, j; f).$$0 MB trainable parameters and no fine-tuning during incremental phases. The ablation on CUB200 attributes the main gain to CLS attention steering, with average accuracy improving from $$\mathcal{L}_{\mathrm{agg}}(\boldsymbol{x}_{1:m}, z; f) = \frac{1}{m} \sum_{i=1}^m \sum_{j=1}^k p(y_i=j\mid\boldsymbol{x}_{1:m}, z)\,\mathcal{L}(\boldsymbol{x}_i, j; f).$$1 for a fully fine-tuned ViT baseline to $$\mathcal{L}_{\mathrm{agg}}(\boldsymbol{x}_{1:m}, z; f) = \frac{1}{m} \sum_{i=1}^m \sum_{j=1}^k p(y_i=j\mid\boldsymbol{x}_{1:m}, z)\,\mathcal{L}(\boldsymbol{x}_i, j; f).$$2 after adding CAGP, then to $$\mathcal{L}_{\mathrm{agg}}(\boldsymbol{x}_{1:m}, z; f) = \frac{1}{m} \sum_{i=1}^m \sum_{j=1}^k p(y_i=j\mid\boldsymbol{x}_{1:m}, z)\,\mathcal{L}(\boldsymbol{x}_i, j; f).$$3 after adding PCAP, CDAP, and MTM (Huang et al., 23 Jan 2026).

The assumptions and limitations are equally formulation-specific. CFAO requires aggregate-label independence from features given labels and conditional independence of labels within a group given features; if these assumptions fail, unbiasedness is no longer guaranteed (Wei et al., 2023). The medical Cls-unbias method is explicitly designed for binary classification, and the paper notes that extension to many classes is non-trivial because few or no features may be informative across all classes (Zuo et al., 9 Aug 2025). CASP may still be challenged by extremely low-shot settings or severe domain shift, and its MTM hyperparameters are dataset-dependent (Huang et al., 23 Jan 2026). Unsupervised debiasing methods depend on the ability of a first-stage model to capture bias structure, while FairKL either requires bias labels or a bias-capturing model to produce bias indicators (Barbano et al., 2022, Barbano et al., 2022).

Taken together, these results show that Class-Unbiased Model (Cls-unbias) is best understood as a technical umbrella for methods that remove class-wise distortion at different points in the learning pipeline: in the statistical risk itself, in the class-wise training dynamics, or in the attention and representation geometry used for classification.