VB-CGCD: Bayesian Category Discovery

• The paper introduces VB-CGCD, a method that employs stochastic variational inference on multivariate Gaussian class-conditionals to balance stability and plasticity in continual learning.
• VB-CGCD models each class’s feature distribution with a Gaussian and uses covariance-aware nearest-class-mean classification along with pseudo-label re-labeling for improved accuracy.
• Experimental results indicate VB-CGCD can boost final accuracy by up to 15.21% on benchmarks such as CIFAR-100, TinyImageNet, ImageNet-100, and CUB while mitigating catastrophic forgetting.

Searching arXiv for the main paper and a few directly related methods to ground the article in current literature. arXiv_search(query="(Dai et al., 23 Jul 2025)", max_results=5, sort_by="relevance") arXiv_search(query="Continual Generalized Category Discovery Learning and Forgetting from a Bayesian Perspective", max_results=10, sort_by="relevance") Variational Bayes C-GCD (VB-CGCD) is a Bayesian framework for Continual Generalized Category Discovery (C-GCD) that models each category’s feature distribution as a multivariate Gaussian and learns these distributions via stochastic variational inference. It is designed for the exemplar-free setting in which a model must incrementally learn new classes from unlabeled data streams while preserving knowledge of old classes, even when unlabeled sessions mix previously seen and novel categories. Its defining components are variational Gaussian estimation, covariance-aware nearest-class-mean classification, re-labeling for pseudo-label correction, and an early-stopping rule that aligns covariance scales across sessions (Dai et al., 23 Jul 2025).

1. Problem setting and scope

C-GCD considers a stream of sessions $t = 0, 1, \ldots, T$. At $t = 0$, a labeled set $D_l$ with $C_l$ known classes is available. In subsequent online sessions $t \ge 1$, the model receives an unlabeled set $D_u^t$ drawn from a mixture of previously seen classes in $D_l$ and novel classes, so that the total number of classes $C_u = C_l + C_n$ grows as sessions progress. The objective is twofold: discover and learn novel categories appearing in the unlabeled stream, and retain performance on old classes, all without accessing past raw data (Dai et al., 23 Jul 2025).

Within this setting, VB-CGCD targets the stability–plasticity problem in a particularly سخت case: unlabeled online data do not separate known from unknown categories in advance. Existing C-GCD methods are described as suffering catastrophic forgetting after learning new classes, with decision boundaries shifting, old-class accuracy degrading, and overall accuracy decreasing over sessions. VB-CGCD is motivated by the claim that these failures can be analyzed from a Bayesian perspective in terms of how class distributions evolve rather than only in terms of prototype drift or parameter interference. This suggests a shift from purely geometric prototype updates toward posterior inference over class-conditional uncertainty.

2. Probabilistic formulation and variational inference

Let $h(x) \in \mathbb{R}^M$ denote the frozen feature extractor output, with the paper giving DINO ViT-B/16 as an example. For each accumulated class $k$, VB-CGCD specifies a multivariate Gaussian class-conditional model

$t = 0$0

where $t = 0$1. A class-posterior classifier is then built from these class-conditionals and class priors $t = 0$2, which may be uniform or proportional to class support in the current session (Dai et al., 23 Jul 2025).

Rather than estimating $t = 0$3 and $t = 0$4 by point statistics, the framework treats them as latent variables and infers their posterior with variational Bayes. If $t = 0$5 collects the parameters of all classes, the prior $t = 0$6 is taken to be Gaussian and initialized as $t = 0$7 for stability and neutrality at the start of a novel class. The variational objective is the evidence lower bound,

$t = 0$8

which, in the per-class MVN setting, reduces to

$t = 0$9

VB-CGCD maximizes $D_l$0 by stochastic variational inference, updating $D_l$1 and $D_l$2 with stochastic gradients. The paper explicitly contrasts this choice with the classic Normal-Inverse-Wishart prior, noting that VB-CGCD instead uses simple Gaussian priors over the latent parameters and performs gradient-based SVI updates for scalability and robustness to outliers (Dai et al., 23 Jul 2025).

For a single data point $D_l$3 assigned to class $D_l$4, the Gaussian likelihood gradients are given as

$D_l$5

and

$D_l$6

Aggregated over minibatches and weighted by pseudo-label responsibilities, these updates are intended to be robust to noisy assignments through the interaction of likelihood and KL regularization.

3. Covariance misalignment, forgetting, and the decision rule

A central claim of VB-CGCD is that catastrophic forgetting in C-GCD is driven by covariance misalignment between old and new classes. With a covariance-aware classifier, the per-class Mahalanobis metric is

$D_l$7

In diagonal form with variances $D_l$8 along dimension $D_l$9, this becomes

$C_l$0

As SVI fits $C_l$1 to new classes, the new-class covariance may expand. When it grows relative to old-class covariances, distances in Mahalanobis space become imbalanced, the decision boundary becomes biased toward the new class, and old decision regions fragment (Dai et al., 23 Jul 2025).

The paper quantifies this effect with the Bhattacharyya distance between two Gaussians,

$C_l$2

When $C_l$3 for a new class significantly exceeds $C_l$4 for old classes, the second log-determinant term grows and the overlap increases asymmetrically, leading to systematic misclassification of old samples. This is the paper’s Bayesian account of forgetting: the classifier acquires an irreversible bias through covariance scale mismatch rather than only through mean displacement.

The deployed classifier is a covariance-aware NCM extended to class-specific covariances, described as a quadratic discriminant. Its decision score is

$C_l$5

with prediction $C_l$6. The $C_l$7 term regularizes preference for overly spread covariances; when priors are equal, the rule reduces to Mahalanobis distance plus a complexity penalty. The paper states that this quadratic discriminant rule is Bayes-optimal under Gaussian class-conditionals (Dai et al., 23 Jul 2025).

4. Session-wise training pipeline

VB-CGCD operates in an offline phase followed by repeated online phases. In the offline session $C_l$8, the backbone $C_l$9 is fine-tuned on $t \ge 1$0 and then frozen for all subsequent sessions. The fine-tuning loss is

$t \ge 1$1

where $t \ge 1$2 is a label-agnostic contrastive loss used to stabilize features. Initial class parameters $t \ge 1$3 for the $t \ge 1$4 known classes are then estimated via SVI on the labeled features and stored (Dai et al., 23 Jul 2025).

For each online session $t \ge 1$5, the pipeline begins by extracting frozen features $t \ge 1$6 for $t \ge 1$7 and clustering them into $t \ge 1$8 tentative novel classes, with k-means given as the example. If the number of novel classes is unknown, the framework can estimate it with silhouette score or other off-the-shelf methods. A first SVI pass fits provisional Gaussians $t \ge 1$9 to these clusters, producing an initial estimate of new-class distributions (Dai et al., 23 Jul 2025).

The next stage is re-labeling, or self-correction. The stored old-class distributions $D_u^t$0 are merged with the provisional novel-class distributions $D_u^t$1 to form a unified classifier, and labels are reassigned by

$D_u^t$2

over old and novel classes. The method may optionally retain only high-confidence assignments satisfying $D_u^t$3. The stated intuition is that samples from old classes are typically closer to stored old Gaussians than to imperfect novel prototypes, while truly novel samples remain closer to the fitted novel Gaussians (Dai et al., 23 Jul 2025).

A second SVI pass then refits Gaussian parameters using the re-labeled data. Covariance alignment is enforced not by directly optimizing

$D_u^t$4

which proved tricky because of competing forces and data asymmetry across sessions, but by an early-stopping criterion

$D_u^t$5

with updates for each novel class halted when $D_u^t$6 and $D_u^t$7. Empirically, the paper reports that stopping when the new-class covariance aligns with the mean determinant of old classes maximizes overall accuracy while preventing irreversible forgetting. Pseudo-label noise suppression is attributed to the ELBO itself, through low reconstruction likelihood for mislabeled samples and a strong KL penalty for latent parameters that deviate to fit outliers (Dai et al., 23 Jul 2025).

5. Experimental results, ablations, and implementation

The standard evaluation protocol uses CIFAR-100, TinyImageNet, ImageNet-100, and CUB-200. In that configuration, 50% of classes are labeled with 80% of training images per labeled class; the remaining 50% of classes are unlabeled and split evenly across $D_u^t$8 online sessions; each session introduces novel classes using 80% of training images per novel class, while 20% of past data are carried forward as unlabeled known-class data. Evaluation at each session reports All, Old, and New accuracies over all classes seen so far. The paper also introduces a more challenging benchmark with only 10% of categories labeled, 90% unlabeled distributed across $D_u^t$9 online sessions, and extended online phases (Dai et al., 23 Jul 2025).

On the standard 50%-labeled, $D_l$0 setting, VB-CGCD achieves an average $D_l$1 improvement in final All accuracy over state-of-the-art baselines across C100, Tiny, IN100, and CUB. On the 10%-labeled, $D_l$2 benchmark, it attains $D_l$3 final All accuracy, compared with a strong baseline identified as Happy at $D_l$4. A plausible implication is that the method’s covariance-aware updates retain utility under both conventional and severely label-scarce continual discovery regimes (Dai et al., 23 Jul 2025).

The ablation study isolates the contribution of each major component. Replacing Mahalanobis distance with Euclidean distance reduces final accuracy by approximately $D_l$5 points on average. SVI yields consistently higher accuracy than point estimates and makes covariance invertible and stable even when there are fewer samples than features; without SVI, covariances can be ill-conditioned. Removing covariance alignment increases new accuracy transiently but worsens overall All accuracy because of old-class forgetting. Re-labeling improves performance by approximately $D_l$6 on large datasets and by approximately $D_l$7 on CUB, where noise ratios are higher (Dai et al., 23 Jul 2025).

Implementation details are explicit. The backbone is DINO ViT-B/16, with DINOv2 ViT-B/14 also tested. Fine-tuning is offline only and uses BOFT for parameter-efficient tuning. Features are reduced by PCA to 384 dimensions. Fine-tuning uses AdamW with learning rate $D_l$8 for 10 epochs; SVI uses a fixed learning rate of $D_l$9 for 1000 update steps per session in the two-pass strategy. Experiments are reported on an RTX 8000 (48GB) with seed 0. Storage is classwise, requiring $C_u = C_l + C_n$0 and a symmetric $C_u = C_l + C_n$1 per class, with the lower triangle stored to halve memory. SVI mini-batches scale linearly in data, and no raw exemplar storage is required (Dai et al., 23 Jul 2025).

6. Relation to adjacent methods, limitations, and future directions

VB-CGCD is situated as a Bayesian continual learning method, but its locus of uncertainty differs from approaches that regularize network parameters. The paper explicitly relates it to variational continual learning methods such as VCL, GVCL, and S-FSVI, while emphasizing that VB-CGCD models class-conditionals directly and analyzes forgetting through covariance dynamics. It is also contrasted with NCM/prototypical classifiers and GMM prompts such as FeCAM and PromptCCD, because it explicitly fits per-class covariances with SVI and uses a Mahalanobis classifier with log-det regularization. Relative to non-Bayesian GCD approaches such as MetaGCD and GM, the paper presents covariance alignment and the early-stopping rule as a principled account of the stability–plasticity trade-off (Dai et al., 23 Jul 2025).

The stated limitations are specific. Small-sample covariance estimation, especially on CUB, remains challenging because per-class sample sizes can be low. Performance depends on the quality of initial clustering, since poor clusters propagate noise to the initial novel distributions. The Gaussian class-conditional assumption may be violated in some domains, and diagonal covariance approximations degrade performance relative to full covariance. These limitations suggest that the method’s strongest performance should be expected when features are sufficiently well-behaved for full-covariance Gaussian modeling to be informative.

The future directions given in the paper remain within the same probabilistic design space: replacing k-means with deep clustering or adaptive mixture models, using hierarchical or empirical Bayes priors to stabilize covariance estimation under limited data, and integrating Bayesian nonparametrics such as Dirichlet processes for unknown class counts. Together, these directions indicate that VB-CGCD is best understood not as a fixed recipe, but as a Bayesian program for continual category discovery in which uncertainty over class geometry is central (Dai et al., 23 Jul 2025).

7. Acronym ambiguity and disambiguation

The shorthand “VB-CGCD” is not standardized across the broader variational Bayes literature. In the continual learning context, it denotes “Variational Bayes C-GCD,” the method described above for Continual Generalized Category Discovery (Dai et al., 23 Jul 2025).

In unrelated literatures, the same shorthand has been used only informally or retrospectively for different methods. In semi-blind graph signal recovery, it has been used to describe a Variational Bayes algorithm whose coordinate update for each adjacency weight yields a generalized compound confluent hypergeometric posterior; the corresponding paper itself studies “Statistical Graph Signal Recovery Using Variational Bayes” rather than a method formally named VB-CGCD (Torkamani et al., 2020). In variational logistic modeling, it has been used as an interpretive label for mean-field variational Bayes in conditionally conjugate exponential-family models solved by coordinate ascent updates, with the Jaakkola–Jordan bound linked formally to Polya–Gamma augmentation; again, the paper does not explicitly define the acronym (Durante et al., 2017). In Gaussian factor modeling, it has likewise been used as a convenience label for “Variational Bayes for Gaussian factor models under the cumulative shrinkage process,” not as the paper’s formal method name (Legramanti, 2020).

This nomenclatural overlap can create a superficial ambiguity. In current continual learning usage, however, VB-CGCD refers specifically to the covariance-aware Bayesian framework for continual generalized category discovery introduced in 2025, not to those unrelated variational Bayes constructions.