Class Distribution Mismatch (CDM)

• CDM is a discrepancy where the class distributions differ between datasets, undermining model performance in various learning settings.
• The literature provides formal definitions, theoretical analyses, and algorithmic strategies like logit adjustment and re-balancing to address this mismatch.
• Empirical results indicate that mitigating CDM enhances balanced accuracy and robustness in semi-supervised, federated, and anomaly detection tasks.

Class distribution mismatch (CDM) is the discrepancy between the class or label distributions in different components of a machine learning system, such as between training versus test sets, labeled versus unlabeled data partitions, or across heterogeneous sites in distributed settings. CDM is a prominent challenge in semi-supervised learning, open-world and federated learning, anomaly detection, online drift monitoring, and numerous application domains where assumptions of identical class priors are violated. This article provides an authoritative survey of the formal definitions, theoretical underpinnings, algorithmic strategies, and empirical insights for CDM, substantiated by recent work.

1. Formal Definitions and Problem Settings

The central notion in CDM is the difference between the reference class prior (typically training set, labeled pool, or initial environment) and the shifted class prior (test-time, unlabeled set, or a new environment). Let $P_\mathrm{ref}(y)$ be the class distribution in the reference (e.g., labeled set) and $P_\mathrm{target}(y)$ be the distribution in the target domain (e.g., test set, unlabeled set, or incoming data stream). CDM occurs when $P_\mathrm{ref}(y) \neq P_\mathrm{target}(y)$.

In semi-supervised learning (SSL), two sets are given:

• Labeled: $\mathcal X = \{ (x_n, y_n) \}_{n=1}^N$ with class counts $N_c$
• Unlabeled: $\mathcal U = \{ u_m \}_{m=1}^M$ with true—but unobserved—class counts $M_c$

Imbalance ratios are defined as $\gamma_l = N_1/N_C$, $\gamma_u = M_1/M_C$ (for classes sorted $N_1 \geq \cdots \geq N_C$). CDM is present whenever $\gamma_l \neq \gamma_u$ or, more generally, when $P_\mathrm{labeled}(y) \neq P_\mathrm{unlabeled}(y)$ (Lee et al., 15 Mar 2024).

In open-world or class-mismatch SSL, the unlabeled set may include both "in-distribution" (ID) classes and additional "out-of-distribution" (OOD) or novel classes absent from the labeled set. CDM is then characterized by partial or non-overlapping support between labeled and unlabeled class sets. Formally, if $D_\ell$ (labeled) supports $Y_\mathrm{ID}$ and $D_u$ (unlabeled) supports $Y_\mathrm{ID} \cup Y_\mathrm{OOD}$ with $Y_\mathrm{OOD} \cap Y_\mathrm{ID} = \emptyset$, CDM holds whenever the OOD fraction $$\rho = | \{ x_j : y_j \in Y_\mathrm{OOD} \} | / |D_u| > 0$$ (Han et al., 2023, Du et al., 2023).

In federated and distributed settings, CDM surfaces as class-prior heterogeneity both across clients (non-i.i.d. label distributions) and within a client’s local labeled and unlabeled pools (Wang et al., 2021). Drift detection and online monitoring frame CDM temporally, as changes in class-conditional or marginal class distributions over a stream (Stucchi et al., 2022). Quantification is often via divergence measures such as KL-divergence, total-variation, or Earth Mover's Distance.

2. Theoretical Analysis and Error Decomposition

Theoretical frameworks decompose the excess risk or population loss under CDM into intrinsic terms and CDM-specific penalties. For SSL under CDM, the population risk $R(\hat h)$ for a classifier $\hat h$ trained on labeled and pseudo-labeled data decomposes into:

• Generalization gap
• Training error
• The CDM-induced SSL error $\mathcal{E}_\mathrm{SSL}$, further split as:
  • Pseudo-labeling error: label error on target-class (ID) data
  • Invasion error: misattribution of OOD samples to ID classes

This is made explicit in (Du et al., 2023), where for sets $T$ (true ID) and $\hat T$ (training, including pseudo-labeled and OOD-data points),

$$\mathcal{E}_\mathrm{SSL} = \left| \frac{1}{|T|} \sum_{T} \ell - \frac{1}{|\hat T|} \sum_{\hat T} \ell \right|$$

which is upper-bounded by a combination of pseudo-labeling and invasion errors. Crucially, robust SSL under CDM requires minimizing both:

• The maximum pointwise mutual information (PMI) alignment between pseudo-labeled and true-class samples (confidence control — controls pseudo-labeling error)
• The expected weight on OOD samples (filtering — controls invasion error)

In federated settings, regularization terms penalizing deviation from uniform class probability and discouraging over-confident softmax outputs are shown to stabilize convergence and improve accuracy under both across-client and within-client CDM scenarios (Wang et al., 2021).

3. Algorithmic Approaches and Correction Strategies

A broad spectrum of algorithmic methodologies has emerged to address CDM. Representative approaches include:

Bias Correction via Logit Adjustment or Post-hoc Calibration:

• CDMAD (Lee et al., 15 Mar 2024) subtracts a bias vector $b = g_\theta(\mathcal{I})$ (logits on a "null" input) from both training and test-time logits, generalizing post-hoc logit adjustment to account for unknown class priors in unlabeled data. This confers Fisher-consistency for the balanced error rate even when unlabeled class distributions are arbitrary.

Re-Balancing and Pseudo-label Filtering:

• Pseudo-labeling (PL), when naively applied, is highly sensitive to OOD contamination, as OOD samples often receive majority-class pseudo-labels, skewing learning (Han et al., 2023). Re-Balanced Pseudo-Labeling (RPL) subsamples exactly $N$ high-confidence pseudo-labels per class to equalize class counts and filter out weakly-aligned OOD data.
• Semantic Exploration Clustering (SEC) uses optimal-transport to assign remaining low-confidence data into extra clusters, approximating semantic OOD grouping and improving OOD utilization.

Density-Driven and Feature-Space Approaches:

• Tailedness proxies via local feature-space density estimates guide dynamic temperature scaling for contrastive loss and adapt pseudo-label margins according to class uncertainty or prototype density, addressing the undefined class prior of unlabeled data (Park et al., 31 May 2024).

Unsupervised Open-World Classifiers via Synthetic Generation:

• UCDM (Du et al., 11 May 2025) constructs positive/negative image pairs by semantic modification via diffusion models (e.g., text-to-image, text-conditional erasure), allowing joint open- and closed-set classifier training with only class-name supervision, no labels.

Robust Ensemble and Thresholding in Anomaly Detection:

• SPADE constructs an ensemble of one-class classifiers and uses partial matching via Wasserstein distances to tune pseudo-label thresholds, thus addressing semi-supervised anomaly detection without assuming class-prior correspondence (Yoon et al., 2022).

Dataset Selection and Quantitative Mismatch Assessment:

• MixMOOD (Calderon-Ramirez et al., 2020) introduces deep dataset dissimilarity measures (DeDiMs)—median-aggregated $\ell_2$, $\ell_1$, Jensen-Shannon divergence, cosine distances in a fixed feature space—to rank candidate unlabeled sets by their similarity to labeled data, empirically predicting SSL task benefit under CDM in MixMatch-style frameworks.

A sample of these methods and the types of CDM they address appears below:

Method / Paper	Key Mechanism	CDM Scenario
CDMAD (Lee et al., 15 Mar 2024)	Logit bias correction	Imbalance, SSL
WAD (Du et al., 2023)	PMI-based pseudo-label filtering	Open-world SSL
UCDM (Du et al., 11 May 2025)	Diffusion-model pair generation	Fully unsupervised
RPL+SEC (Han et al., 2023)	Pseudo-label balancing, clustering	OOD, SSL mismatch
SPADE (Yoon et al., 2022)	OCC ensemble, partial matching	Anomaly, SSL
Fed-SHVR (Wang et al., 2021)	Confidence regularizers, norm. avg	Federated, SSL
MixMOOD (Calderon-Ramirez et al., 2020)	Distance-based dataset ranking	Any, pre-selection
SNU_IDS (Bae et al., 2019)	Loss reweight, thresholding, sampling	Training–Test shift

4. Empirical Findings, Benchmarks, and Practical Insights

Empirical studies across benchmarks reveal several important phenomena:

• Pseudo-labeling and classical SSL collapse under increasing OOD contamination, with accuracy dropping below the supervised-only baseline on common SSL datasets at high mismatch (Han et al., 2023, Du et al., 2023).
• Quantitative measures (KL divergence, max/min ratio of pseudo-label frequencies) concretely diagnose skew induced by OOD data.
• Corrective methods such as CDMAD yield substantial gains in balanced accuracy (e.g., FixMatch+CDMAD vs. FixMatch: 87.5% vs. 68.9% on CIFAR-10-LT under severe mismatch (Lee et al., 15 Mar 2024)).
• Unsupervised UCDM surpasses SSL baselines requiring thousands of labels, even with 0 labels, as measured by open-set and closed-set accuracy (Du et al., 11 May 2025).
• In federated settings, robust penalty-based algorithms maintain high accuracy as mismatch ratios increase, and save hundreds of communication rounds relative to naïve FL baselines (Wang et al., 2021).
• Feature-based dataset selection (MixMOOD) strongly correlates with SSL performance, producing monotonic improvement as feature-space distance decreases (correlation $\approx -0.94$ for cosine distance) (Calderon-Ramirez et al., 2020).

A plausible implication is that all practical SSL, open-world, and federated learning deployments benefit from explicit CDM modeling, and ante-hoc dataset analysis for data selection. Empirical ablations consistently show that removing balancing, correction, or open-set components substantially degrades accuracy, especially on tail and novel classes.

5. Implementation Principles and Model Architectures

Effectively addressing CDM requires architectural, loss, and pipeline modifications:

• Loss modifications: Weighted cross-entropy with class weights $w^c = P_\mathrm{target}(c) / P_\mathrm{ref}(c)$ (Bae et al., 2019); regularizers penalizing overconfident pseudo-labels (Wang et al., 2021); logit or score calibration at inference.
• Open- and closed-set outputs: Multi-head networks for open-set detection, e.g., separate sigmoid and softmax heads (Du et al., 11 May 2025).
• Progressive and selective pseudo-labeling: Ranking, confidence thresholds, expansion of labeled pool with high-confidence samples, and feature-based semantic clustering (Han et al., 2023, Du et al., 2023).
• Distributionally aware sampling: Mini-batch or overall data resampling to match estimated test priors.
• Ensembling: Bagging with resampled or weighted mini-batches, or OCC-based consensus (Yoon et al., 2022).

Pipeline variants are validated via balanced accuracy, open-set metrics, false-alarm rate (for drift), or communication efficiency (for FL).

6. Theoretical Guarantees and Practical Limitations

Theoretical analyses supply Fisher consistency for balanced error under logit-correction schemes (Lee et al., 15 Mar 2024), tight high-probability upper bounds on excess risk in terms of mutual information and OOD sample weights (Du et al., 2023), and principled convergence rates under CDM even in distributed communication-limited regimes (Wang et al., 2021).

Limitations commonly noted:

• Many approaches assume knowledge or estimability of some class prior in unlabeled data (enabling better correction).
• Diffusion-based instance generation (UCDM) incurs substantial compute.
• Some clustering or balancing strategies scale poorly or are sensitive to hyperparameters (SEC).
• Existing methods are generally tuned for image classification; extensions to other modalities remain ongoing.

Future work includes more efficient open-set augmentation, automated prompt engineering for diffusion models, and explicit uncertainty quantification in multi-modal or streaming settings.

7. Applications, Evaluation, and Benchmarks

CDM arises in:

• Semi-supervised and open-world learning on natural images, text, or medical data
• Online concept drift and class-conditional drift detection in evolving datastreams (Stucchi et al., 2022)
• Semi-supervised anomaly and fraud detection with changing class or anomaly priors (Yoon et al., 2022)
• Federated learning with client heterogeneity and partial label annotation (Wang et al., 2021)
• Data selection and curation for semi-supervised pipelines (Calderon-Ramirez et al., 2020)

Standard practice involves generating synthetic splits with controlled mismatch, measuring errors on both ID and OOD instances, and comparing against both supervised and SSL baselines, with added ablations to isolate effects attributable to the CDM.

---

CDM represents a core challenge in robust and generalizable learning. Recent algorithmic, theoretical, and empirical advances have substantially improved both the understanding and mitigation of distributional mismatches across diverse machine learning paradigms.