Continual Generalized Category Discovery (C-GCD)

• Continual GCD is a continual learning setting that extends static GCD by processing sequential unlabeled sessions to recognize both existing and novel classes.
• It tackles catastrophic forgetting by balancing stability and plasticity, while mitigating biases such as prediction and hardness biases.
• Representative methods leverage pretrained backbones, contrastive learning, and fixed classifier geometries to maintain coherent feature representations over time.

Continual Generalized Category Discovery (C-GCD) is a continual learning setting in which a model is initially trained on labeled data from known classes and then receives a sequence of unlabeled sessions containing a mixture of previously seen and genuinely novel categories. The task is to recognize old classes, discover new classes, and expand the classifier over time without retraining from scratch on all past data. It extends static Generalized Category Discovery (GCD), which assumes a single labeled–unlabeled mixture, to sequential data streams, thereby combining open-world category discovery with the stability–plasticity constraints of continual learning (Vaze et al., 2022, Wu et al., 2023).

1. Historical emergence and problem scope

The static precursor, Generalized Category Discovery, is defined on a labeled set $\mathcal{D_L}$ and an unlabeled set $\mathcal{D_U}$ with $\mathcal{Y_L} \subset \mathcal{Y_U}$, so that unlabeled data contain both old and new classes. Its objective is to categorize all images in $\mathcal{D_U}$ without assuming that the number of novel classes is known a priori (Vaze et al., 2022). C-GCD inherits this mixed known–unknown structure but imposes a sequential constraint: unlabeled data arrive in multiple sessions, the test set grows over time, and performance on previously learned categories must be retained.

MetaGCD explicitly names this setting “Continual Generalized Category Discovery (C-GCD)” and formalizes it as a sequence of sessions $$\{\mathcal{S}^0,\mathcal{S}^1,\ldots,\mathcal{S}^T\}$$, where $\mathcal{S}^0$ is labeled and later sessions are unlabeled mixtures of known and novel classes (Wu et al., 2023). In parallel, OpenGCD formulates an open-world recognition pipeline $[\mathcal{O},\mathcal{L},\mathcal{I}]$ that sequentially performs unknown detection, generalized category discovery, and incremental learning; although it is not presented under the C-GCD name, it operationalizes a closely related multi-phase setting with exemplars and human correction (Gao et al., 2023).

Relative to adjacent paradigms, C-GCD differs in three decisive ways. Unlike class-incremental learning, later sessions are unlabeled. Unlike open-set or open-world recognition, unknowns must be partitioned into semantic classes rather than merely rejected. Unlike novel category discovery, the unlabeled stream contains both old and new classes in every session, so the model must simultaneously preserve existing decision structure and discover unseen categories (Wu et al., 2023, Ma et al., 2024).

2. Formal setting, notation, and evaluation

A common formulation begins with a labeled dataset

$D_l=\{(x_i^l,y_i^l)\}$

containing $C_l$ known classes, and an unlabeled stream

$D_u=\{D_u^t\}_{t=1}^T,\qquad D_u^t=\{x_i^u\},$

where the total number of classes in the stream is $\mathcal{D_U}$0, with $\mathcal{D_U}$1 novel classes (Dai et al., 23 Jul 2025). Each session mixes samples from previously seen classes and newly appearing classes. After session $\mathcal{D_U}$2, the model is evaluated on all classes encountered up to that point.

The most widely reported per-session metrics are $\mathcal{D_U}$3, $\mathcal{D_U}$4, and $\mathcal{D_U}$5: accuracy on all seen classes, on old classes only, and on classes newly introduced in the current session. Hungarian matching is typically used to align predicted cluster indices with semantic labels. Several papers also summarize long-horizon behavior with forgetting and discovery scores, although the exact formulas vary. VB-CGCD defines forgetting rate $\mathcal{D_U}$6 as the drop in accuracy on the initial labeled classes and novelty learning rate $\mathcal{D_U}$7 as the average of $\mathcal{D_U}$8 across sessions, whereas GOAL and Happy report $\mathcal{D_U}$9 and $\mathcal{Y_L} \subset \mathcal{Y_U}$0 following the GM-style continual discovery protocol (Dai et al., 23 Jul 2025, Han et al., 23 Feb 2026).

Architecturally, most recent methods adopt a pretrained Vision Transformer backbone, usually DINO or DINOv2, and freeze most of the encoder during continual stages. The learned object is then not only a feature extractor but also a mechanism for discovering new clusters, calibrating old-versus-new assignments, and retaining class structure without raw-data replay. This suggests that in C-GCD, representation stability is as central as classifier expansion.

3. Core technical challenges

The defining challenge is catastrophic forgetting under unlabeled mixed streams. MetaGCD frames the problem as one of discovering novel classes while maintaining performance on known classes without access to previous sessions (Wu et al., 2023). Later work refines this diagnosis. Happy identifies two systematic biases: prediction bias, in which the model allocates excessive probability mass to old classes because they were learned under full supervision, and hardness bias, in which difficult old classes are forgotten more severely than easy ones because their prototypes are closer to competing classes in feature space (Ma et al., 2024).

VB-CGCD provides a different analysis by modeling each class in feature space as a Gaussian and examining continual learning through a Bayesian lens. In that view, forgetting is driven by covariance misalignment between old and new class distributions. Under a diagonal covariance assumption, classification depends on a Mahalanobis metric,

$\mathcal{Y_L} \subset \mathcal{Y_U}$1

and distributional separation is analyzed with the Bhattacharyya distance between old and new class Gaussians. The paper identifies three phases—stability, critical transition, and forgetting—and argues that once the determinant of a new-class covariance grows substantially larger than those of old classes, the decision boundary skews toward the new class and old-class accuracy degrades irreversibly (Dai et al., 23 Jul 2025).

GOAL diagnoses a related but geometrically framed instability: dynamic classifier weights drift over time, so old-class directions move when new classes are integrated. Its response is to impose a fixed simplex Equiangular Tight Frame (ETF) over the total class set and to align all features to that geometry throughout training. This makes old class directions stationary and treats continual discovery as a feature-alignment problem rather than a repeated redefinition of classifier geometry (Han et al., 23 Feb 2026).

NCENet approaches the same difficulty from representation learning. It argues that incremental unlabeled adaptation must improve novel-class discrimination while preserving old-class structure, and therefore splits its solution into Neighborhood Commonality-aware Representation Learning for plasticity and Bi-level Contrastive Knowledge Distillation for stability (Wang et al., 2024). Across these analyses, a common conclusion emerges: C-GCD failure is not reducible to simple parameter drift; it is tied to miscalibrated probabilities, unstable prototype geometry, and representation updates that distort old class neighborhoods.

4. Methodological families and representative systems

The C-GCD literature has developed several method families that differ in where they encode stability: in the initialization, in the classifier geometry, in prototypes, or in explicit probabilistic class models.

Method	Core mechanism	Distinctive emphasis
MetaGCD	Meta-learning with soft neighborhood contrastive adaptation	Offline simulation of future continual discovery
Happy	Clustering-guided initialization, soft entropy regularization, hardness-aware prototype sampling	Prediction bias and hardness bias
NCENet	NCRL + BCKD	Representation evolution without replay
VB-CGCD	Variational Bayes MVN modeling + covariance-aware NCM	Bayesian analysis of forgetting
GOAL	Fixed ETF classifier with supervised and confidence-guided alignment	Stable global geometry
OpenGCD	OSR + GCD + exemplar-based IL	Open-world pipeline with human correction

MetaGCD uses a bi-level optimization scheme. During meta-training, labeled data are split into pseudo-base and pseudo-incremental sessions; the inner loop adapts on unlabeled data with a soft neighborhood contrastive loss,

$\mathcal{Y_L} \subset \mathcal{Y_U}$2

while the outer loop evaluates the adapted parameters on accumulated pseudo-test classes with supervised contrastive loss. The aim is to learn an initialization explicitly optimized for “discover without forgetting” (Wu et al., 2023).

Happy retains a parametric classifier but debiases it. New classifier heads are initialized from cluster centroids that are least similar to old heads, soft entropy regularization controls marginal probability mass across old and new groups, and old-class retention is handled by hardness-aware sampling from Gaussian class prototypes plus feature-level distillation. This makes prototype replay selective rather than class-uniform (Ma et al., 2024).

NCENet dispenses with explicit pseudo-labeling during training. It computes per-sample local commonality vectors from batch $\mathcal{Y_L} \subset \mathcal{Y_U}$3-nearest neighbors,

$\mathcal{Y_L} \subset \mathcal{Y_U}$4

uses these to form prediction distributions across two views, and distills them with

$\mathcal{Y_L} \subset \mathcal{Y_U}$5

Old knowledge is preserved by a symmetric contrastive distillation loss between teacher and student features (Wang et al., 2024).

VB-CGCD treats C-GCD as Bayesian continual clustering–classification. Each class is modeled as a multivariate normal in feature space, posterior inference is approximated with stochastic variational inference, and classification is performed by a covariance-aware nearest-class-mean rule: $\mathcal{Y_L} \subset \mathcal{Y_U}$6 The distinctive mechanism is covariance-driven early stopping, which halts updates when the determinant of a new covariance approaches the average determinant of old covariances, preventing covariance blow-up (Dai et al., 23 Jul 2025).

GOAL relocates stability into fixed classifier geometry. It constructs a simplex ETF $\mathcal{Y_L} \subset \mathcal{Y_U}$7, uses supervised alignment in the base session,

$\mathcal{Y_L} \subset \mathcal{Y_U}$8

and later performs confidence-guided unsupervised alignment of novel clusters to previously unused ETF directions. The classifier can expand, but the target geometric structure remains fixed (Han et al., 23 Feb 2026).

5. Benchmarks and empirical landscape

The standard benchmarks are CIFAR-100, TinyImageNet, ImageNet-100, and CUB-200-2011, with stage-0 using labeled data from roughly half the classes and later sessions containing unlabeled mixtures of old and new classes. Several papers also study longer horizons, such as 10-stage protocols, and harsher label scarcity, such as settings with only 10% labeled classes (Ma et al., 2024, Dai et al., 23 Jul 2025).

MetaGCD established early strong results on CIFAR10, CIFAR100, and Tiny-ImageNet, outperforming re-run NCD and continual discovery baselines under C-GCD constraints. On the final session it reports improvements over GM in New accuracy of $\mathcal{Y_L} \subset \mathcal{Y_U}$9 on CIFAR10, $\mathcal{D_U}$0 on CIFAR100, and $\mathcal{D_U}$1 on Tiny-ImageNet, and improvements over VanillaGCD in All accuracy of $\mathcal{D_U}$2, $\mathcal{D_U}$3, and $\mathcal{D_U}$4, respectively (Wu et al., 2023).

Happy emphasizes a more practical rehearsal-free long-horizon setting with more new classes and more stages. It reports “remarkable performance across various datasets, e.g., 7.5% overall gains on ImageNet-100,” and shows lower forgetting $\mathcal{D_U}$5 together with higher discovery $\mathcal{D_U}$6 than MetaGCD and other baselines on CIFAR-100 and Tiny-ImageNet (Ma et al., 2024).

NCENet further improves representation quality in this regime. On CIFAR100 in the last incremental learning session, its clustering accuracy exceeds the second-best method by $\mathcal{D_U}$7 on old classes and $\mathcal{D_U}$8 on new classes; on Tiny-ImageNet it reports final-session gains of $\mathcal{D_U}$9 All, $$\{\mathcal{S}^0,\mathcal{S}^1,\ldots,\mathcal{S}^T\}$$0 Old, and $$\{\mathcal{S}^0,\mathcal{S}^1,\ldots,\mathcal{S}^T\}$$1 New over MetaGCD (Wang et al., 2024).

VB-CGCD reports the strongest gains among the methods summarized here. On the standard 5-session setup it reports final-session All accuracies of $$\{\mathcal{S}^0,\mathcal{S}^1,\ldots,\mathcal{S}^T\}$$2 on CIFAR-100, $$\{\mathcal{S}^0,\mathcal{S}^1,\ldots,\mathcal{S}^T\}$$3 on Tiny, $$\{\mathcal{S}^0,\mathcal{S}^1,\ldots,\mathcal{S}^T\}$$4 on IN100, and $$\{\mathcal{S}^0,\mathcal{S}^1,\ldots,\mathcal{S}^T\}$$5 on CUB, with an average improvement of $$\{\mathcal{S}^0,\mathcal{S}^1,\ldots,\mathcal{S}^T\}$$6 in final-session overall accuracy over prior art. It also introduces a more challenging benchmark with only $$\{\mathcal{S}^0,\mathcal{S}^1,\ldots,\mathcal{S}^T\}$$7 labeled data and extended online phases, on which it reports $$\{\mathcal{S}^0,\mathcal{S}^1,\ldots,\mathcal{S}^T\}$$8 final accuracy versus $$\{\mathcal{S}^0,\mathcal{S}^1,\ldots,\mathcal{S}^T\}$$9 for the state of the art (Dai et al., 23 Jul 2025).

GOAL targets long-horizon stability rather than Bayesian modeling. Against Happy it reports average $\mathcal{S}^0$0 and $\mathcal{S}^0$1, compared with $\mathcal{S}^0$2 and $\mathcal{S}^0$3 for Happy, corresponding to a $\mathcal{S}^0$4 reduction in forgetting and a $\mathcal{S}^0$5 improvement in discovery. In 10-stage experiments on CIFAR100 and TinyImageNet it further reports large reductions in forgetting together with higher All and New accuracies (Han et al., 23 Feb 2026).

6. Limitations, unresolved issues, and future directions

Several limitations recur across the literature. First, many methods assume that the number of novel classes per session is known; some papers provide estimation strategies, but with some accuracy loss. Static GCD already treated class-count estimation as a difficult black-box optimization problem, and this difficulty persists in continual settings (Vaze et al., 2022, Dai et al., 23 Jul 2025). OpenGCD addresses this with class-number estimation for semi-supervised $\mathcal{S}^0$6-means, but its overall pipeline still requires human correction and semantic labeling of novel clusters, so it does not fully automate C-GCD (Gao et al., 2023).

Second, many systems depend on strong frozen backbones such as DINO or DINOv2 ViTs. This suggests that a substantial fraction of current progress comes from pretrained representation quality rather than from continual discovery machinery alone. It also leaves domain shift insufficiently addressed: NCENet explicitly notes that heavy domain shift across sessions is not studied, and VB-CGCD likewise assumes standard in-distribution splits rather than severe inter-session shift (Wang et al., 2024, Dai et al., 23 Jul 2025).

Third, storage and scalability remain contested. OpenGCD uses a fixed-size, class-balanced exemplar memory chosen by DS3, whereas Happy, GOAL, NCENet, MetaGCD, and VB-CGCD are presented as rehearsal-free or exemplar-free in the sense that they do not store raw old images. Yet prototype storage, Gaussian statistics, covariance matrices, or frozen teacher models remain part of the state. VB-CGCD notes that full covariances scale quadratically with feature dimension and therefore uses PCA to reduce dimension to 384 and also studies a diagonal covariance variant (Gao et al., 2023, Dai et al., 23 Jul 2025).

Finally, the field still differs on what the principal bottleneck is. MetaGCD emphasizes initialization and offline task simulation; Happy emphasizes debiasing; NCENet emphasizes neighborhood structure and contrastive distillation; VB-CGCD emphasizes covariance dynamics; GOAL emphasizes fixed classifier geometry. This suggests not a single settled theory of C-GCD failure, but a convergence around three recurring design requirements: robust novel-class discovery from unlabeled mixed streams, explicit control of old-class drift, and a representation or classifier geometry that remains coherent as the label space expands.

A plausible implication is that future progress will come from hybridizing these views rather than choosing among them. The literature already points toward adaptive class-count estimation, hierarchical or shared Bayesian priors for small-sample covariance estimation, multimodal cues for prototype assignment, and geometric structures that remain stable under long-horizon continual discovery (Dai et al., 23 Jul 2025, Han et al., 23 Feb 2026).