Novel Class Discovery in ML

Updated 10 July 2026

Novel Class Discovery is the process of using labeled data to infer unseen, disjoint categories in unlabeled datasets across various domains.
Methodologies include two-stage and one-stage approaches that employ pseudo-labeling, optimal transport, and consistency regularization for effective knowledge transfer.
Key challenges involve managing class imbalance, addressing realistic evaluation protocols, and preventing label leakage to ensure robust discovery.

Novel Class Discovery (NCD) is the problem of inferring novel categories in an unlabeled set by leveraging prior knowledge from a labeled set containing different, but related classes. In the canonical formulation, the labeled set $L=\{(x_i,y_i)\}$ is drawn from known classes $C_L$ , the unlabeled set $U=\{x_j\}$ is drawn from novel classes $C_U$ , and $C_L \cap C_U=\emptyset$ (Troisemaine et al., 2023). The field has expanded from image classification to semantic segmentation, heterogeneous tabular data, 3D point cloud semantic segmentation, node classification on graphs, and multi-view data, while also motivating related settings such as generalized category discovery, class-incremental NCD, and open-world graph learning (Zhao et al., 2021, Troisemaine et al., 2022, Riz et al., 2023, Hou et al., 2024, Wan et al., 16 Jul 2025).

1. Formal setting and task boundaries

The defining assumption of NCD is class disjointness between labeled and unlabeled data. This separates it from semi-supervised learning, where labeled and unlabeled samples share the same classes, and from open-set or open-world recognition, where the main objective is often to detect unknowns rather than partition them into multiple new categories. In the multi-view formulation, NCD is also distinguished from conventional multi-view clustering because it explicitly leverages labeled known classes and focuses on clustering unseen classes rather than learning a purely unsupervised consensus partition (Wan et al., 16 Jul 2025, Troisemaine et al., 2023).

The standard objective is to discover the structure of $U$ while transferring useful information from $L$ . In some formulations, the output space remains split between known and novel classes; in others, a unified classifier is trained over $C_L \cup C_U$ . This distinction underlies the common separation between task-aware evaluation, where the system is told whether a sample belongs to the known or novel subset, and task-agnostic evaluation, where no such indicator is available at test time (Wang et al., 2024, Fini et al., 2021).

Several closely related extensions relax the classical setting. Open-world graph learning allows unlabeled nodes to contain both known and novel classes, formalized as $\mathcal{Y}^L \subset \mathcal{Y}^U$ (Jin et al., 2024). Continual variants introduce sequential sessions in which new unlabeled classes arrive over time and previously seen data are unavailable, forcing the model to balance discovery with preservation of earlier knowledge (Chen et al., 2024, Yu et al., 2024, Hou et al., 2024). These variants preserve the central NCD idea—knowledge transfer from known to unknown classes—but alter the assumptions about label availability, temporal access, and test-time ambiguity.

2. Main methodological families

The NCD literature has converged on two broad families: two-stage approaches and one-stage approaches. Two-stage methods first extract knowledge from labeled data only and then apply it to unlabeled data. Representative examples include learned pairwise similarity transfer, latent-space clustering, and expert-based variants such as Meta Discovery. One-stage methods instead jointly optimize supervised and unsupervised objectives on labeled and unlabeled sets, typically through a shared encoder and separate known/novel heads (Troisemaine et al., 2023).

A canonical one-stage formulation is UNO, which introduces a unified cross-entropy objective over the concatenated label space. Its loss is

$\ell(\mathbf{x},\mathbf{y}) = - \sum_{c=1}^C \mathbf{y}_c \log(\mathbf{p}_c),$

with padded targets

$C_L$ 0

Pseudo-labels for $C_L$ 1 are produced by an entropy-regularized optimal transport problem solved with Sinkhorn-Knopp, and the same objective is applied homogeneously to ground-truth labels and pseudo-labels (Fini et al., 2021). This unified treatment is characteristic of the one-stage trend.

Subsequent work has increasingly regularized how knowledge is transferred. “Modeling Inter-Class and Intra-Class Constraints in Novel Class Discovery” introduces inter-class and intra-class constraints based on symmetric Kullback–Leibler divergence, explicitly using the disjointness of labeled and unlabeled classes to enforce separability while constraining consistency across augmentations (Li et al., 2022). “Class-relation Knowledge Distillation” preserves a class-relation vector over known classes for novel samples, regularizing discovery by matching the student’s known-class predictions on novel data to those of a teacher trained on known classes (Gu et al., 2023). “Self-Cooperation Knowledge Distillation” goes further by constructing two disjoint representation spaces for known and novel classes and coupling them through a spatial mutual-information mechanism and bidirectional distillation losses (Wang et al., 2024). Collectively, these methods move beyond a purely shared embedding and treat knowledge transfer itself as an explicit modeling object.

3. Pseudo-labeling, cluster cardinality, and realistic assumptions

Pseudo-label generation is a central mechanism in NCD. The survey literature groups common strategies into cosine-threshold binarization, RankStats based on top- $C_L$ 2 feature indices, WTA hashing, and balanced optimal-transport assignments (Troisemaine et al., 2023). In UNO, Sinkhorn-Knopp enforces balanced cluster assignments within a mini-batch, while swapped prediction across two augmented views stabilizes learning (Fini et al., 2021). In image segmentation, EUMS ranks images by foreground entropy, splits them into clean and unclean subsets, dynamically reassigns part of the clean set after five epochs, and applies teacher-student self-training on the unclean split (Zhao et al., 2021). In dermoscopy, uncertainty-aware multi-view cross pseudo-supervision weights cross-view supervision by a KL-based uncertainty term and further refines pseudo-labels by aggregating neighborhood information from a memory bank (Feng et al., 2023).

A recurring issue is the assumption that the number of novel classes is known in advance. This assumption is explicit in many classification and segmentation formulations, including UNO, IIC, SCKD, and NCDSS (Fini et al., 2021, Li et al., 2022, Wang et al., 2024, Zhao et al., 2021). The tabular study “A Practical Approach to Novel Class Discovery in Tabular Data” argues that such assumptions, together with the use of novel labels for hyperparameter tuning, are unrealistic in real-world scenarios. It proposes adapting $C_L$ 3-fold cross-validation by hiding some known classes in each fold, and reports that the latent space of its simple deep model can be used to reliably estimate the number of novel classes (Troisemaine et al., 2023).

Imbalance further complicates pseudo-labeling. “Novel Class Discovery for Long-tailed Recognition” shows that equal-size optimal transport, as used in standard self-labeling NCD, can force head-class points into minority clusters under long-tailed distributions. Its remedy is a relaxed optimal transport problem coupled with an equiangular prototype representation, so that pseudo-label assignment no longer assumes class-balanced novel categories (Zhang et al., 2023). This suggests that pseudo-label design is not merely an auxiliary component; it determines whether the inductive bias supplied by known classes is helpful or systematically misleading.

4. Domain-specific formulations

In tabular data, the absence of spatial inductive bias and the presence of heterogeneous variables require substantially different machinery from vision. TabularNCD combines VIME self-supervised pretraining, a shared MLP encoder, a classification head over known classes plus an additional “novel aggregate” class, and a clustering head trained from top- $C_L$ 4 cosine-neighbor pseudo-labels in latent space (Troisemaine et al., 2022). A later practical formulation for tabular data emphasizes realistic model selection without any prior knowledge of the novel classes and also adapts $C_L$ 5-means and Spectral Clustering to leverage the knowledge of known classes (Troisemaine et al., 2023). An interactive system extends this line of work to expert-in-the-loop usage by exposing TabularNCD, $C_L$ 6-means, spectral clustering, latent T-SNE views, and decision-tree rules for interpretation (Troisemaine et al., 2023).

Dense prediction has prompted separate formulations. Novel Class Discovery in Semantic Segmentation introduces NCDSS, where the challenges are object/background separation, the presence of multiple classes within an image, and noisy pseudo-labels. Its basic framework combines a base segmentation model with saliency to isolate novel foreground, then clusters features to create pseudo-labels, while EUMS adds entropy-based uncertainty modeling and self-training; on PASCAL-5 $C_L$ 7, the basic framework reaches an average mIoU of $C_L$ 8, and EUMS improves it by $C_L$ 9 (Zhao et al., 2021). For 3D point clouds, NOPS argues that transposing the 2D method is suboptimal and instead uses uncertainty-aware online clustering with entropic optimal transport and prototypes for pseudo-labeling, improving novel-class mIoU by $U=\{x_j\}$ 0 on SemanticPOSS and $U=\{x_j\}$ 1 on SemanticKITTI over the adapted 2D baseline (Riz et al., 2023). In indoor RGB-D scenes, NeurNCD replaces explicit 3D segmentation maps with an implicit semantic field built through Embedding-NeRF and a KL consistency loss between pre-trained semantic embeddings and rendered semantic distributions, reporting overall mIoU $U=\{x_j\}$ 2 on NYUv2 and known/novel mIoU $U=\{x_j\}$ 3 on Replica without labels in the target scenes (Wang et al., 6 Jun 2025).

Graph and multi-view settings introduce additional structure. ORAL addresses open-world graph learning by clustering trainable prototypes, deriving group-aware attention weights, and refining graph structure with pseudo-labels, achieving the best “All” accuracy across Cora, BlogCatalog, and AmazonPhoto in the reported comparisons (Jin et al., 2024). NC-NCD frames node classification as a stage-wise setting without access to old labeled nodes during discovery, and SWORD combines self-training with prototype replay and feature distillation to preserve old classes while clustering new nodes (Hou et al., 2024). Multi-view NCD is first explored by IICMVNCD, which couples intra-view matrix factorization with inter-view weight guidance from known classes and avoids pseudo-label bootstrapping for unlabeled data (Wan et al., 16 Jul 2025). Continual formulations such as ADM and FEA extend the same agenda to incremental sessions, where the central issues become catastrophic forgetting, inter-session confusion, and parameter-efficient model merging (Chen et al., 2024, Yu et al., 2024).

5. Evaluation protocols and benchmarks

The most common evaluation metric in classification-style NCD is clustering accuracy after Hungarian matching,

$U=\{x_j\}$ 4

often reported together with NMI and ARI (Wang et al., 2024). Task-aware evaluation measures known and novel performance separately with access to the subset identity at test time, whereas task-agnostic evaluation requires prediction over the combined label space without such hints (Fini et al., 2021, Li et al., 2022). In segmentation, the dominant metric is mIoU on novel classes, sometimes alongside base-class or overall mIoU (Zhao et al., 2021, Riz et al., 2023). Continual graph formulations additionally report $U=\{x_j\}$ 5, $U=\{x_j\}$ 6, $U=\{x_j\}$ 7, and harmonic means to capture the trade-off between discovery and retention (Hou et al., 2024).

Setting	Typical evaluation	Representative benchmarks
Image classification NCD	ACC, NMI, ARI; task-aware and task-agnostic protocols	CIFAR10, CIFAR100, ImageNet (Fini et al., 2021, Li et al., 2022, Wang et al., 2024)
Semantic segmentation	mIoU on novel classes, and often base/overall mIoU	PASCAL-5 $U=\{x_j\}$ 8, COCO-20 $U=\{x_j\}$ 9 (Zhao et al., 2021)
3D point cloud segmentation	mIoU on novel, base, and all classes	SemanticKITTI, SemanticPOSS (Riz et al., 2023)
Tabular NCD	ACC, BACC, NMI, ARI	Seven public classification datasets (Troisemaine et al., 2022)
Node and graph settings	$C_U$ 0, $C_U$ 1, $C_U$ 2, harmonic mean; or open-world ACC on known/novel/all	Cora, Citeseer, Pubmed, Wiki-CS, AmazonPhoto, BlogCatalog (Hou et al., 2024, Jin et al., 2024)
Multi-view NCD	ACC, NMI, Purity	BRCA, KIPAN, Cora, STL10, YTB10 (Wan et al., 16 Jul 2025)

Benchmark design itself has become part of the problem definition. “A Closer Look at Novel Class Discovery from the Labeled Set” argues that earlier benchmarks largely ignored the semantic relation between labeled and unlabeled sets, and constructs ImageNet and CIFAR100 splits with controlled high-, medium-, and low-similarity regimes (Li et al., 2022). This benchmark perspective shifts evaluation away from raw class counts and toward the quality of transfer that the labeled set can actually provide.

6. Central findings and open research questions

One of the clearest findings in the literature is that the labeled set is not uniformly beneficial. The labeled-set study introduces transfer leakage as a semantic similarity measure and shows that it highly correlates with NCD performance. More strikingly, it reports that using labels may lead to sub-optimal outcomes in low-similarity settings (Li et al., 2022). This challenges a widespread assumption in early NCD work and places labeled-set design on equal footing with objective design.

A second persistent issue is imbalance. SCKD identifies a long-neglected trade-off between reviewing known classes and discovering novel classes when the numbers of known and novel samples are imbalanced, and it addresses this by using two disjoint representation spaces with bidirectional distillation (Wang et al., 2024). Long-tailed NCD sharpens the same point: equal-size pseudo-label constraints can be fundamentally mismatched to realistic class distributions, and prototype geometry plus relaxed OT can yield substantial gains on ImageNet100, Herbarium19, and iNaturalist18 (Zhang et al., 2023).

A third line of tension concerns temporal and structural realism. Class-incremental NCD, continual NCD, and NC-NCD all report catastrophic forgetting, inter-session confusion, or the inability to balance old and new categories as defining obstacles (Chen et al., 2024, Yu et al., 2024, Hou et al., 2024). In parallel, domain-specific work repeatedly shows that assumptions imported from image classification or 2D segmentation do not transfer unchanged to tabular data, graphs, or 3D scenes (Troisemaine et al., 2023, Riz et al., 2023, Jin et al., 2024). A plausible implication is that NCD is no longer well described as a single benchmark problem; it is a family of discovery problems whose shared core is class-disjoint transfer, but whose operational constraints are domain-dependent.

Current work therefore converges on a more restrictive notion of realism: no leakage of novel labels into tuning, careful treatment of cluster-number uncertainty, robustness to imbalance, and architectures that preserve known-class competence while exposing novel structure. The tabular results showing that NCD can be solved without relying on knowledge from the novel classes support that direction directly (Troisemaine et al., 2023). This suggests that future progress will be judged less by performance on class-balanced image splits alone than by how well a method retains these properties under heterogeneous data types, open-world ambiguity, and continual deployment.