Mixed Bias Class in Machine Learning

• Mixed Bias Class is a category of class-dependent bias where inherent class heterogeneity is intertwined with graph topology, spurious attributes, and pseudo-label dynamics.
• It manifests across domains such as graph neural networks, image classification, medical diagnosis, and imbalanced semi-supervised learning, challenging the assumption of homogeneous class behavior.
• Mitigation strategies include neutral calibration, pairwise intrinsic-feature enhancement, and group-level balancing, all yielding significant empirical improvements.

Searching arXiv for the cited papers to ground the article and verify metadata. arXiv search query: "NeuBM Mitigating Model Bias in Graph Neural Networks through Neutral Input Calibration" Mixed Bias Class denotes a family of class-dependent bias phenomena in which bias is not uniform across classes, or even within a single class, but is entangled with additional structure such as graph topology, spurious attributes, latent subgroups, pseudo-label dynamics, or selection effects. The current literature suggests that the term is not fully standardized and is instead used for related formulations across several subfields. In graph learning, it refers to the joint effect of class imbalance, topology imbalance, and label scarcity on logits and representations (Gu et al., 21 May 2025). In image classification, it refers to classes containing both bias-aligned and bias-conflicting samples (Park et al., 2024). In medical diagnosis, it refers to the simultaneous presence of class-feature bias and class imbalance (Zuo et al., 9 Aug 2025). In imbalanced semi-supervised learning, it refers to the coexistence of model-side bias from long-tailed labeled data and pseudo-label bias on unlabeled data (Li et al., 2023, Wang et al., 2022). In group-robust classification, it can denote multimodal class-conditional bias distributions that require group-level rather than class-level balancing (Zhao et al., 11 May 2025).

1. Terminological scope and definitions

The literature uses Mixed Bias Class to describe several structurally similar, but not identical, phenomena. What unifies these formulations is that class bias is coupled to another axis of heterogeneity, so that a class cannot be treated as a homogeneous statistical object.

Domain	Mixed bias formulation	Representative paper
Graph neural networks	Class imbalance combined with graph topology and limited labels	(Gu et al., 21 May 2025)
Image classification	A class contains both bias-aligned and bias-conflicting samples	(Park et al., 2024)
Medical diagnosis	Class-feature bias together with class imbalance	(Zuo et al., 9 Aug 2025)
Imbalanced SSL	Labeled-side model bias and unlabeled-side pseudo-label bias	(Li et al., 2023)
Group-robust classification	Multimodal $p(b \mid y)$ with latent groups	(Zhao et al., 11 May 2025)
UDA object detection	Receptive fields contain mixed categories, creating mixed-class features	(Gou et al., 2022)

In the GNN setting, mixed bias class is described as the combined bias from class imbalance and graph topology, amplified by label scarcity, which produces skewed decision boundaries and majority-dominated representations (Gu et al., 21 May 2025). In image classification, the same class may contain bias-aligned samples that carry a spurious cue and bias-conflicting samples that lack or contradict it, so the class itself is internally heterogeneous (Park et al., 2024). In medical diagnosis, the relevant distinction is whether a feature is informative for one subset of classes but uninformative for another, which creates class-feature bias; when this coincides with class imbalance, the resulting problem is explicitly called mixed bias (Zuo et al., 9 Aug 2025).

The term also appears in semi-supervised learning in a more procedural sense. There, the first source of bias is the imbalanced labeled set, which distorts the classifier, and the second source is the pseudo-label distribution on unlabeled data, which inherits and amplifies that distortion (Wang et al., 2022, Li et al., 2023). In group-robust classification, mixed bias class refers to the case where the class-conditional bias distribution is multi-modal, so a single class contains several distinct spurious modes, formalized as $$p(b \mid y) = \sum_{k=1}^{K} \pi_{y,k} p(b \mid y, z=k)$$ (Zhao et al., 11 May 2025).

2. Generative mechanisms of mixed bias

A central mechanism is asymmetric signal allocation. In imbalanced GNN training, cross-entropy exposes the model more often to majority-class errors, message passing spreads majority signals through neighborhoods, and sparse minority labels provide weak supervision. The result is not merely a skewed class prior but a structural preference for majority-class topology and feature subspaces (Gu et al., 21 May 2025). In this formulation, class bias is inseparable from graph structure.

A second mechanism is within-class shortcut heterogeneity. In bias-contrastive image classification, samples from one class do not share a single bias state. Some are bias-aligned and contain the spuriously correlated attribute, whereas others are bias-conflicting. Because intrinsic class-defining features are expected to be common across both, while spurious features differ, the problem becomes one of isolating class-discerning common features rather than globally suppressing a known bias attribute (Park et al., 2024). This shifts the target from coarse reweighting toward pairwise feature comparison within class.

A third mechanism is loss asymmetry across classes. In the medical diagnosis formulation, class-feature bias is diagnosed by inequalities in class-wise cross-entropy losses $L^{pos}$ and $L^{neg}$. Persistent discrepancy between them indicates that the model is exploiting class-specific features that do not transfer across classes, and under imbalance this effect is exacerbated because underperforming classes contribute less reliably to training (Zuo et al., 9 Aug 2025). A related mechanism appears in imbalanced SSL, where unknown unlabeled class distributions and confidence-thresholded pseudo-labeling create a feedback loop: head classes receive more confident pseudo-labels, which further improves head-class fit and suppresses tail-class recovery (Li et al., 2023).

A fourth mechanism is mode collapse in bias modeling. Original class-conditional distribution balancing methods approximate $p(b \mid y)$ with a single Gaussian, but if the class-conditional bias distribution is multi-modal, that approximation conflates distinct spurious modes and leaves residual dependence between $b$ and $y$ after reweighting. Fine-grained formulations therefore replace single-distribution alignment with latent-group alignment (Zhao et al., 11 May 2025). In object detection, an analogous problem occurs spatially: feature receptive fields may contain multiple object classes, so image-level or pixel-level alignment can inadvertently align single-class features to mixed-class features, producing negative transfer (Gou et al., 2022).

3. Manifestations in predictions, representations, and diagnostics

Mixed bias class manifests at the level of logits, decision boundaries, and feature geometry. In biased GNNs, majority-class logits become systematically larger, decision boundaries shift toward minority regions, and learned node representations cluster more tightly around majority-class centroids. The NeuBM analysis further states that calibration reduces class discrepancy in a representation-balancing sense, with $$\mathrm{MMD}_{\mathrm{NeuBM}}(c_1,c_2) < \mathrm{MMD}(c_1,c_2)$$, linking output calibration to class-conditional feature alignment (Gu et al., 21 May 2025).

In image models, the phenomenon is visible in attribution maps. The bias-contrastive method identifies class-discerning common features and amplifies under-exploited intrinsic regions by applying a spatial guidance map $g(z)=z \odot IE(z)$, where $IE(z)$ combines commonality and relative under-exploitation. The paper reports that this guidance suppresses attention to spurious regions such as water, sky, and background while shifting Grad-CAM toward intrinsic regions such as wings, beaks, body parts, or human action (Park et al., 2024).

A related but sharper diagnostic appears in binary attribute classification. On CelebA, unbalanced training produced a striking asymmetry: majority-class predictions often showed little activation in expected attribute regions and instead relied on image corners, diffuse global regions, or the bias neuron $$p(b \mid y) = \sum_{k=1}^{K} \pi_{y,k} p(b \mid y, z=k)$$0 in the final layer, whereas minority-class predictions were more localized and overlapped better with expected facial regions. Balanced training using attribute-specific class weights made majority and minority activations more semantically plausible and reduced the asymmetry in false negative and false positive rates (Zhang et al., 2024).

The literature also records a useful corrective to a common assumption. In decision trees trained on imbalanced data, the longstanding belief that the model is necessarily biased toward the majority class does not always hold. Under specific conditions related to the predictors, trees fit to purity with only one positive case are biased toward the minority class, and split-level bias persists with more than one positive case (Phelps et al., 9 Jan 2025). This does not contradict the existence of majority-class neglect in other models; it shows that bias direction depends on the data-generating process, the split criterion, and the definition of bias.

4. Mitigation strategies

One major strategy is post-hoc or in-training calibration against a neutral reference. NeuBM constructs a class-agnostic neutral graph $$p(b \mid y) = \sum_{k=1}^{K} \pi_{y,k} p(b \mid y, z=k)$$1 using the average node count $$p(b \mid y) = \sum_{k=1}^{K} \pi_{y,k} p(b \mid y, z=k)$$2, average edge density $$p(b \mid y) = \sum_{k=1}^{K} \pi_{y,k} p(b \mid y, z=k)$$3, and node-feature statistics $$p(b \mid y) = \sum_{k=1}^{K} \pi_{y,k} p(b \mid y, z=k)$$4. The calibrated logits are $$p(b \mid y) = \sum_{k=1}^{K} \pi_{y,k} p(b \mid y, z=k)$$5, followed by $$p(b \mid y) = \sum_{k=1}^{K} \pi_{y,k} p(b \mid y, z=k)$$6. The method adds one extra forward pass, with reported runtime around $$p(b \mid y) = \sum_{k=1}^{K} \pi_{y,k} p(b \mid y, z=k)$$7–$$p(b \mid y) = \sum_{k=1}^{K} \pi_{y,k} p(b \mid y, z=k)$$8 baseline and memory overhead of $$p(b \mid y) = \sum_{k=1}^{K} \pi_{y,k} p(b \mid y, z=k)$$9–$L^{pos}$0, and it is reported to work with GCN, GAT, GraphSAGE, GIN, and Graph Transformers (Gu et al., 21 May 2025).

A second strategy is pairwise intrinsic-feature enhancement within class. The bias-contrastive framework trains a biased model $L^{pos}$1 and a debiased model $L^{pos}$2, constructs a same-class auxiliary sample $L^{pos}$3 that is likely bias-conflicting using a bias-negative score, and computes a common feature score $L^{pos}$4, a relative-exploitation score $L^{pos}$5, and an enhancement map $L^{pos}$6. Training combines a reweighted main classification loss, a guidance loss with $L^{pos}$7, and a BN loss, while inference uses $L^{pos}$8 alone (Park et al., 2024).

A third strategy is explicit equalization of class-wise losses. In medical diagnosis, the class-wise inequality loss is $L^{pos}$9, and it is combined with a smoothed class-wise group DRO objective $L^{neg}$0, where $L^{neg}$1 is obtained by a stop-gradient softmax over class losses. The total objective is $L^{neg}$2, with schedules in which $L^{neg}$3 increases and $L^{neg}$4 decreases during training (Zuo et al., 9 Aug 2025).

Imbalanced SSL introduces two additional families of corrections. The Bias Adaptive Classifier decomposes prediction into an unbiased linear classifier and a removable residual bias attractor, $L^{neg}$5, and trains them in a bi-level fashion so that the attractor absorbs imbalance bias while the linear head is optimized on a balanced upper-level objective (Wang et al., 2022). TCBC instead performs two corrections: a model-side prior correction using $L^{neg}$6, and a pseudo-label-side refinement using $L^{neg}$7, thereby targeting both the labeled and unlabeled sources of class bias (Li et al., 2023).

Several methods operate at the data or subgroup level rather than directly on logits. Bias Mimicking constructs class-conditioned subsampled datasets $L^{neg}$8 such that $L^{neg}$9 for all $p(b \mid y)$0; Proposition 1 shows that this implies $p(b \mid y)$1 within $p(b \mid y)$2. The procedure trains a debiased encoder across all such datasets without repeating samples within an epoch (Qraitem et al., 2022). Fine-grained CCDB uses Bias Exploration via Overfitting to infer pseudo-bias groups and then assigns group-level weights $p(b \mid y)$3, which are converted into per-sample weights $p(b \mid y)$4, thereby matching reweighted $p(b \mid y)$5 to the marginal $p(b \mid y)$6 (Zhao et al., 11 May 2025). DCAST addresses unidentified class-specific selection bias by pseudo-labeling per class, selecting a candidate pool of size $p(b \mid y)$7, clustering it, and retaining the most confident sample per cluster, so that diversity is enforced at selection time rather than through an auxiliary loss (Tepeli et al., 2024).

In domain-adaptive object detection, the mitigation target is mixed-class misalignment itself. SCFAM defines a mixed-classes $p(b \mid y)$8-divergence $p(b \mid y)$9 and builds local, mid-level, and global semantic prediction modules with semantic bridging components. The total objective combines the standard detector loss with adversarial domain losses, semantic prediction losses, and semantic consistency regularization, $b$0 (Gou et al., 2022).

5. Empirical evidence

Across GNN benchmarks, neutral calibration yields large gains when imbalance is severe. Reported macro-F1 improvements include Cora with $b$1, where NeuBM reaches $b$2 versus $b$3 for GCN and $b$4 for TAM, Amazon Computers with $b$5, where NeuBM reaches $b$6 versus $b$7 for GCN and $b$8 for TAM, DBLP, where NeuBM reaches $b$9 versus $y$0 for GCN, and PubMed, where NeuBM reaches $y$1 versus $y$2 for GAT. On Cora, the smallest class’s F1 improves by $y$3 over TAM, and the majority–minority F1 gap decreases from $y$4 to $y$5 (Gu et al., 21 May 2025).

In mixed-bias image classification, the pairwise intrinsic-feature method reports state-of-the-art results across synthetic and real datasets. On Waterbirds, average class accuracy reaches $y$6 at $y$7 versus $y$8 for LfF+BE; on BFFHQ, BC accuracy reaches $y$9 at $$\mathrm{MMD}_{\mathrm{NeuBM}}(c_1,c_2) < \mathrm{MMD}(c_1,c_2)$$0 versus $$\mathrm{MMD}_{\mathrm{NeuBM}}(c_1,c_2) < \mathrm{MMD}(c_1,c_2)$$1 for LfF+BE; and on BAR, average accuracy reaches $$\mathrm{MMD}_{\mathrm{NeuBM}}(c_1,c_2) < \mathrm{MMD}(c_1,c_2)$$2 at $$\mathrm{MMD}_{\mathrm{NeuBM}}(c_1,c_2) < \mathrm{MMD}(c_1,c_2)$$3 versus $$\mathrm{MMD}_{\mathrm{NeuBM}}(c_1,c_2) < \mathrm{MMD}(c_1,c_2)$$4 for LfF+BE. The paper also reports that selecting $$\mathrm{MMD}_{\mathrm{NeuBM}}(c_1,c_2) < \mathrm{MMD}(c_1,c_2)$$5 from $$\mathrm{MMD}_{\mathrm{NeuBM}}(c_1,c_2) < \mathrm{MMD}(c_1,c_2)$$6, then $$\mathrm{MMD}_{\mathrm{NeuBM}}(c_1,c_2) < \mathrm{MMD}(c_1,c_2)$$7, then $$\mathrm{MMD}_{\mathrm{NeuBM}}(c_1,c_2) < \mathrm{MMD}(c_1,c_2)$$8 progressively improves performance, and that removing either the guidance loss or the BN loss degrades results (Park et al., 2024).

For mixed bias in medical and diagnostic settings, the class-unbias objective improves macro-F1 across speech and imaging datasets. On DAIC-WOZ, Cls-unbias reaches $$\mathrm{MMD}_{\mathrm{NeuBM}}(c_1,c_2) < \mathrm{MMD}(c_1,c_2)$$9 versus $g(z)=z \odot IE(z)$0 for ERM (cls-w); on MODMA it reaches $g(z)=z \odot IE(z)$1 versus $g(z)=z \odot IE(z)$2; on ADReSS it reaches $g(z)=z \odot IE(z)$3 versus $g(z)=z \odot IE(z)$4; on BreastMNIST it reaches $g(z)=z \odot IE(z)$5 versus $g(z)=z \odot IE(z)$6; and on RetinaMNIST it reaches $g(z)=z \odot IE(z)$7 versus $g(z)=z \odot IE(z)$8 (Zuo et al., 9 Aug 2025).

Imbalanced SSL results show comparable patterns. BAC improves balanced accuracy and geometric mean under both matched and mismatched imbalance ratios; for example, on CIFAR-10 with $g(z)=z \odot IE(z)$9 and uniform unlabeled data, BAC+FixMatch reaches balanced accuracy around $IE(z)$0 versus $IE(z)$1 for the baseline, and geometric mean around $IE(z)$2 versus $IE(z)$3 (Wang et al., 2022). TCBC reports $IE(z)$4 on CIFAR-10-LT with $IE(z)$5, $IE(z)$6, $IE(z)$7, $IE(z)$8, compared with $IE(z)$9 for FixMatch and $$p(b \mid y) = \sum_{k=1}^{K} \pi_{y,k} p(b \mid y, z=k)$$00 for L2AC, and $$p(b \mid y) = \sum_{k=1}^{K} \pi_{y,k} p(b \mid y, z=k)$$01 on STL10-LT compared with $$p(b \mid y) = \sum_{k=1}^{K} \pi_{y,k} p(b \mid y, z=k)$$02 for L2AC (Li et al., 2023).

Fine-grained group balancing and diverse self-training also show broad improvements. FG-CCDB reaches worst-group accuracy $$p(b \mid y) = \sum_{k=1}^{K} \pi_{y,k} p(b \mid y, z=k)$$03 on CelebA versus $$p(b \mid y) = \sum_{k=1}^{K} \pi_{y,k} p(b \mid y, z=k)$$04 for CCDB and $$p(b \mid y) = \sum_{k=1}^{K} \pi_{y,k} p(b \mid y, z=k)$$05 on CivilComments versus $$p(b \mid y) = \sum_{k=1}^{K} \pi_{y,k} p(b \mid y, z=k)$$06, while on cMNIST it improves from $$p(b \mid y) = \sum_{k=1}^{K} \pi_{y,k} p(b \mid y, z=k)$$07 to $$p(b \mid y) = \sum_{k=1}^{K} \pi_{y,k} p(b \mid y, z=k)$$08 at $$p(b \mid y) = \sum_{k=1}^{K} \pi_{y,k} p(b \mid y, z=k)$$09 conflicting samples (Zhao et al., 11 May 2025). Under hierarchy bias, DCAST with neural networks significantly improves over supervised learning on $$p(b \mid y) = \sum_{k=1}^{K} \pi_{y,k} p(b \mid y, z=k)$$10–$$p(b \mid y) = \sum_{k=1}^{K} \pi_{y,k} p(b \mid y, z=k)$$11 of $$p(b \mid y) = \sum_{k=1}^{K} \pi_{y,k} p(b \mid y, z=k)$$12 datasets, whereas conventional self-training is significantly worse than supervised learning on $$p(b \mid y) = \sum_{k=1}^{K} \pi_{y,k} p(b \mid y, z=k)$$13 of $$p(b \mid y) = \sum_{k=1}^{K} \pi_{y,k} p(b \mid y, z=k)$$14 datasets; the advantage is reported to be largest on multi-class classification (Tepeli et al., 2024).

6. Related theoretical extensions and limitations

A distinct theoretical usage appears in semiparametric inference, where the Mixed Bias Class is not about class labels at all but about a class of parameters satisfying the mixed bias property. There, the one-step remainder equals the expectation of a product of nuisance-estimation errors,

$$p(b \mid y) = \sum_{k=1}^{K} \pi_{y,k} p(b \mid y, z=k)$$15

and the efficient influence function has the canonical form

$$p(b \mid y) = \sum_{k=1}^{K} \pi_{y,k} p(b \mid y, z=k)$$16

This structure yields rate double robustness, because asymptotic normality follows when the product of nuisance estimation errors is $$p(b \mid y) = \sum_{k=1}^{K} \pi_{y,k} p(b \mid y, z=k)$$17 under cross-fitting (Rotnitzky et al., 2019). A later note proves that algebraic identities between one-step, outcome-regression-type, and IPW-type estimators extend from a strict subclass to the full mixed bias class and can allow one nuisance estimator to be non-linear (Rotnitzky et al., 26 Sep 2025).

Another separate usage arises in cosmology. In mixed dark matter cosmologies with cold dark matter and massive neutrinos, halo bias becomes component-dependent because halos form from the CDM+baryon fluid while neutrinos free-stream on small scales. The effective halo bias with respect to total matter,

$$p(b \mid y) = \sum_{k=1}^{K} \pi_{y,k} p(b \mid y, z=k)$$18

develops a small step-like feature near the neutrino free-streaming scale $$p(b \mid y) = \sum_{k=1}^{K} \pi_{y,k} p(b \mid y, z=k)$$19, with amplitude increasing with halo mass and neutrino fraction (Loverde, 2014). This is conceptually different from algorithmic fairness or class imbalance, but it shows that mixed bias can also denote component-wise response in a multi-source system.

The machine-learning literature also states several practical limitations. NeuBM assumes that sampling from $$p(b \mid y) = \sum_{k=1}^{K} \pi_{y,k} p(b \mid y, z=k)$$20 is a reasonable class-agnostic neutralization strategy; extremely large graphs increase overhead, and dynamic or heterogeneous graphs may require frequent refresh of neutral statistics (Gu et al., 21 May 2025). Bias-contrastive intrinsic enhancement is spatial and can degrade when intrinsic and spurious cues are co-located or when residual bias-aligned contamination remains in the auxiliary set (Park et al., 2024). Class-wise loss equalization is described as most effective in binary or few-class diagnosis, and enforcing strict equality in many-class settings may suppress useful discriminative structure (Zuo et al., 9 Aug 2025). Bias Mimicking requires bias-group labels and assumes mutually exclusive groups; extreme skew can limit feasible retention in the class-conditioned linear program (Qraitem et al., 2022).

These limits suggest that Mixed Bias Class is best understood not as a single algorithmic problem but as a recurrent pattern: class bias becomes “mixed” whenever the operative nuisance is heterogeneous within classes, uneven across groups, or propagated through training dynamics that standard class balancing does not model explicitly.