Papers
Topics
Authors
Recent
Search
2000 character limit reached

VB-CGCD: Bayesian Category Discovery

Updated 14 July 2026
  • The paper introduces VB-CGCD, a method that employs stochastic variational inference on multivariate Gaussian class-conditionals to balance stability and plasticity in continual learning.
  • VB-CGCD models each class’s feature distribution with a Gaussian and uses covariance-aware nearest-class-mean classification along with pseudo-label re-labeling for improved accuracy.
  • Experimental results indicate VB-CGCD can boost final accuracy by up to 15.21% on benchmarks such as CIFAR-100, TinyImageNet, ImageNet-100, and CUB while mitigating catastrophic forgetting.

Searching arXiv for the main paper and a few directly related methods to ground the article in current literature. arXiv_search(query="(Dai et al., 23 Jul 2025)", max_results=5, sort_by="relevance") arXiv_search(query="Continual Generalized Category Discovery Learning and Forgetting from a Bayesian Perspective", max_results=10, sort_by="relevance") Variational Bayes C-GCD (VB-CGCD) is a Bayesian framework for Continual Generalized Category Discovery (C-GCD) that models each category’s feature distribution as a multivariate Gaussian and learns these distributions via stochastic variational inference. It is designed for the exemplar-free setting in which a model must incrementally learn new classes from unlabeled data streams while preserving knowledge of old classes, even when unlabeled sessions mix previously seen and novel categories. Its defining components are variational Gaussian estimation, covariance-aware nearest-class-mean classification, re-labeling for pseudo-label correction, and an early-stopping rule that aligns covariance scales across sessions (Dai et al., 23 Jul 2025).

1. Problem setting and scope

C-GCD considers a stream of sessions t=0,1,…,Tt = 0, 1, \ldots, T. At t=0t = 0, a labeled set DlD_l with ClC_l known classes is available. In subsequent online sessions t≥1t \ge 1, the model receives an unlabeled set DutD_u^t drawn from a mixture of previously seen classes in DlD_l and novel classes, so that the total number of classes Cu=Cl+CnC_u = C_l + C_n grows as sessions progress. The objective is twofold: discover and learn novel categories appearing in the unlabeled stream, and retain performance on old classes, all without accessing past raw data (Dai et al., 23 Jul 2025).

Within this setting, VB-CGCD targets the stability–plasticity problem in a particularly سخت case: unlabeled online data do not separate known from unknown categories in advance. Existing C-GCD methods are described as suffering catastrophic forgetting after learning new classes, with decision boundaries shifting, old-class accuracy degrading, and overall accuracy decreasing over sessions. VB-CGCD is motivated by the claim that these failures can be analyzed from a Bayesian perspective in terms of how class distributions evolve rather than only in terms of prototype drift or parameter interference. This suggests a shift from purely geometric prototype updates toward posterior inference over class-conditional uncertainty.

2. Probabilistic formulation and variational inference

Let h(x)∈RMh(x) \in \mathbb{R}^M denote the frozen feature extractor output, with the paper giving DINO ViT-B/16 as an example. For each accumulated class kk, VB-CGCD specifies a multivariate Gaussian class-conditional model

p(h(x)∣y=k,θk)=N(h(x)∣μk,Σk),p(h(x)\mid y=k,\theta_k)=\mathcal{N}(h(x)\mid \mu_k,\Sigma_k),

where θk={μk,Σk}\theta_k=\{\mu_k,\Sigma_k\}. A class-posterior classifier is then built from these class-conditionals and class priors πk\pi_k, which may be uniform or proportional to class support in the current session (Dai et al., 23 Jul 2025).

Rather than estimating μk\mu_k and Σk\Sigma_k by point statistics, the framework treats them as latent variables and infers their posterior with variational Bayes. If z={μ1,Σ1,…,μK,ΣK}z=\{\mu_1,\Sigma_1,\ldots,\mu_K,\Sigma_K\} collects the parameters of all classes, the prior pΦ(z)p_\Phi(z) is taken to be Gaussian and initialized as N(0,I)\mathcal{N}(0,I) for stability and neutrality at the start of a novel class. The variational objective is the evidence lower bound,

L(q)=Eq(Z,Θ)[log⁡p(X,Z,Θ)]−Eq(Z,Θ)[log⁡q(Z,Θ)],L(q)=\mathbb{E}_{q(Z,\Theta)}[\log p(X,Z,\Theta)]-\mathbb{E}_{q(Z,\Theta)}[\log q(Z,\Theta)],

which, in the per-class MVN setting, reduces to

LELBO=Eq(z∣X)[log⁡pΦ(X∣z)]−DKL(q(z∣X)∥pΦ(z)).L_{\mathrm{ELBO}}=\mathbb{E}_{q(z\mid X)}[\log p_\Phi(X\mid z)]-D_{\mathrm{KL}}(q(z\mid X)\|p_\Phi(z)).

VB-CGCD maximizes LELBOL_{\mathrm{ELBO}} by stochastic variational inference, updating μk\mu_k and Σk\Sigma_k with stochastic gradients. The paper explicitly contrasts this choice with the classic Normal-Inverse-Wishart prior, noting that VB-CGCD instead uses simple Gaussian priors over the latent parameters and performs gradient-based SVI updates for scalability and robustness to outliers (Dai et al., 23 Jul 2025).

For a single data point xx assigned to class kk, the Gaussian likelihood gradients are given as

∂log⁡N(x∣μk,Σk)∂μk=Σk−1(x−μk),\frac{\partial \log \mathcal{N}(x\mid \mu_k,\Sigma_k)}{\partial \mu_k} = \Sigma_k^{-1}(x-\mu_k),

and

∂log⁡N(x∣μk,Σk)∂Σk=12Σk−1[(x−μk)(x−μk)⊤−Σk]Σk−1.\frac{\partial \log \mathcal{N}(x\mid \mu_k,\Sigma_k)}{\partial \Sigma_k} =\frac{1}{2}\Sigma_k^{-1}\big[(x-\mu_k)(x-\mu_k)^\top-\Sigma_k\big]\Sigma_k^{-1}.

Aggregated over minibatches and weighted by pseudo-label responsibilities, these updates are intended to be robust to noisy assignments through the interaction of likelihood and KL regularization.

3. Covariance misalignment, forgetting, and the decision rule

A central claim of VB-CGCD is that catastrophic forgetting in C-GCD is driven by covariance misalignment between old and new classes. With a covariance-aware classifier, the per-class Mahalanobis metric is

dk(x)∝(x−μk)⊤Σk−1(x−μk).d_k(x)\propto (x-\mu_k)^\top \Sigma_k^{-1}(x-\mu_k).

In diagonal form with variances σkj\sigma_k^j along dimension jj, this becomes

dk(x)∝∑j=1M(xj−μkj)2(σkj)2.d_k(x)\propto \sum_{j=1}^M \frac{(x^j-\mu_k^j)^2}{(\sigma_k^j)^2}.

As SVI fits Σk\Sigma_k to new classes, the new-class covariance may expand. When it grows relative to old-class covariances, distances in Mahalanobis space become imbalanced, the decision boundary becomes biased toward the new class, and old decision regions fragment (Dai et al., 23 Jul 2025).

The paper quantifies this effect with the Bhattacharyya distance between two Gaussians,

dB(p1,p2)=18(μ2−μ1)⊤[12(Σ1+Σ2)]−1(μ2−μ1)+12log⁡det⁡[12(Σ1+Σ2)]det⁡Σ1det⁡Σ2.d_B(p_1,p_2)=\frac{1}{8}(\mu_2-\mu_1)^\top \left[\frac{1}{2}(\Sigma_1+\Sigma_2)\right]^{-1}(\mu_2-\mu_1) +\frac{1}{2}\log \frac{\det[\frac{1}{2}(\Sigma_1+\Sigma_2)]}{\sqrt{\det\Sigma_1\det\Sigma_2}}.

When det⁡Σk\det \Sigma_k for a new class significantly exceeds det⁡Σi\det \Sigma_i for old classes, the second log-determinant term grows and the overlap increases asymmetrically, leading to systematic misclassification of old samples. This is the paper’s Bayesian account of forgetting: the classifier acquires an irreversible bias through covariance scale mismatch rather than only through mean displacement.

The deployed classifier is a covariance-aware NCM extended to class-specific covariances, described as a quadratic discriminant. Its decision score is

dk(x)=(h(x)−μk)⊤Σk−1(h(x)−μk)+log⁡∣Σk∣−2log⁡πk,d_k(x)=(h(x)-\mu_k)^\top \Sigma_k^{-1}(h(x)-\mu_k)+\log |\Sigma_k|-2\log \pi_k,

with prediction y^=arg⁡min⁡kdk(x)\hat y=\arg\min_k d_k(x). The log⁡∣Σk∣\log |\Sigma_k| term regularizes preference for overly spread covariances; when priors are equal, the rule reduces to Mahalanobis distance plus a complexity penalty. The paper states that this quadratic discriminant rule is Bayes-optimal under Gaussian class-conditionals (Dai et al., 23 Jul 2025).

4. Session-wise training pipeline

VB-CGCD operates in an offline phase followed by repeated online phases. In the offline session t=0t=0, the backbone h(⋅)h(\cdot) is fine-tuned on DlD_l and then frozen for all subsequent sessions. The fine-tuning loss is

Lft=(1−λ)Lce+λLcon,L_{\mathrm{ft}}=(1-\lambda)L_{\mathrm{ce}}+\lambda L_{\mathrm{con}},

where LconL_{\mathrm{con}} is a label-agnostic contrastive loss used to stabilize features. Initial class parameters θk={μk,Σk}\theta_k=\{\mu_k,\Sigma_k\} for the ClC_l known classes are then estimated via SVI on the labeled features and stored (Dai et al., 23 Jul 2025).

For each online session t≥1t \ge 1, the pipeline begins by extracting frozen features h(x)h(x) for x∈Dutx\in D_u^t and clustering them into CntC_n^t tentative novel classes, with k-means given as the example. If the number of novel classes is unknown, the framework can estimate it with silhouette score or other off-the-shelf methods. A first SVI pass fits provisional Gaussians N(μk′,Σk′)\mathcal{N}(\mu_k',\Sigma_k') to these clusters, producing an initial estimate of new-class distributions (Dai et al., 23 Jul 2025).

The next stage is re-labeling, or self-correction. The stored old-class distributions {N(μi,Σi)}i=1k−1\{\mathcal{N}(\mu_i,\Sigma_i)\}_{i=1}^{k-1} are merged with the provisional novel-class distributions {N(μk′,Σk′)}\{\mathcal{N}(\mu_k',\Sigma_k')\} to form a unified classifier, and labels are reassigned by

q(y=c∣x)=softmaxc{−dc(x)},q(y=c\mid x)=\mathrm{softmax}_c\{-d_c(x)\},

over old and novel classes. The method may optionally retain only high-confidence assignments satisfying max⁡cq(y=c∣x)≥τ\max_c q(y=c\mid x)\ge \tau. The stated intuition is that samples from old classes are typically closer to stored old Gaussians than to imperfect novel prototypes, while truly novel samples remain closer to the fitted novel Gaussians (Dai et al., 23 Jul 2025).

A second SVI pass then refits Gaussian parameters using the re-labeled data. Covariance alignment is enforced not by directly optimizing

Ldet=[log⁡det⁡(Σk)−log⁡(1k−1∑i=1k−1det⁡Σi)]2,L_{\mathrm{det}}=\left[\log \det(\Sigma_k)-\log\left(\frac{1}{k-1}\sum_{i=1}^{k-1}\det \Sigma_i\right)\right]^2,

which proved tricky because of competing forces and data asymmetry across sessions, but by an early-stopping criterion

R=log⁡[(k−1)det⁡(Σk)∑i=1k−1det⁡(Σi)],\mathcal{R}=\log \left[\frac{(k-1)\det(\Sigma_k)}{\sum_{i=1}^{k-1}\det(\Sigma_i)}\right],

with updates for each novel class halted when R<ϵ\mathcal{R}<\epsilon and ϵ≈0.01\epsilon \approx 0.01. Empirically, the paper reports that stopping when the new-class covariance aligns with the mean determinant of old classes maximizes overall accuracy while preventing irreversible forgetting. Pseudo-label noise suppression is attributed to the ELBO itself, through low reconstruction likelihood for mislabeled samples and a strong KL penalty for latent parameters that deviate to fit outliers (Dai et al., 23 Jul 2025).

5. Experimental results, ablations, and implementation

The standard evaluation protocol uses CIFAR-100, TinyImageNet, ImageNet-100, and CUB-200. In that configuration, 50% of classes are labeled with 80% of training images per labeled class; the remaining 50% of classes are unlabeled and split evenly across T=5T=5 online sessions; each session introduces novel classes using 80% of training images per novel class, while 20% of past data are carried forward as unlabeled known-class data. Evaluation at each session reports All, Old, and New accuracies over all classes seen so far. The paper also introduces a more challenging benchmark with only 10% of categories labeled, 90% unlabeled distributed across $9$ online sessions, and extended online phases (Dai et al., 23 Jul 2025).

On the standard 50%-labeled, T=5T=5 setting, VB-CGCD achieves an average +15.21%+15.21\% improvement in final All accuracy over state-of-the-art baselines across C100, Tiny, IN100, and CUB. On the 10%-labeled, T=9T=9 benchmark, it attains 67.86%67.86\% final All accuracy, compared with a strong baseline identified as Happy at 38.55%38.55\%. A plausible implication is that the method’s covariance-aware updates retain utility under both conventional and severely label-scarce continual discovery regimes (Dai et al., 23 Jul 2025).

The ablation study isolates the contribution of each major component. Replacing Mahalanobis distance with Euclidean distance reduces final accuracy by approximately $5.56$ points on average. SVI yields consistently higher accuracy than point estimates and makes covariance invertible and stable even when there are fewer samples than features; without SVI, covariances can be ill-conditioned. Removing covariance alignment increases new accuracy transiently but worsens overall All accuracy because of old-class forgetting. Re-labeling improves performance by approximately 1%1\% on large datasets and by approximately 3%3\% on CUB, where noise ratios are higher (Dai et al., 23 Jul 2025).

Implementation details are explicit. The backbone is DINO ViT-B/16, with DINOv2 ViT-B/14 also tested. Fine-tuning is offline only and uses BOFT for parameter-efficient tuning. Features are reduced by PCA to 384 dimensions. Fine-tuning uses AdamW with learning rate 10−310^{-3} for 10 epochs; SVI uses a fixed learning rate of 10−510^{-5} for 1000 update steps per session in the two-pass strategy. Experiments are reported on an RTX 8000 (48GB) with seed 0. Storage is classwise, requiring μk\mu_k and a symmetric Σk\Sigma_k per class, with the lower triangle stored to halve memory. SVI mini-batches scale linearly in data, and no raw exemplar storage is required (Dai et al., 23 Jul 2025).

6. Relation to adjacent methods, limitations, and future directions

VB-CGCD is situated as a Bayesian continual learning method, but its locus of uncertainty differs from approaches that regularize network parameters. The paper explicitly relates it to variational continual learning methods such as VCL, GVCL, and S-FSVI, while emphasizing that VB-CGCD models class-conditionals directly and analyzes forgetting through covariance dynamics. It is also contrasted with NCM/prototypical classifiers and GMM prompts such as FeCAM and PromptCCD, because it explicitly fits per-class covariances with SVI and uses a Mahalanobis classifier with log-det regularization. Relative to non-Bayesian GCD approaches such as MetaGCD and GM, the paper presents covariance alignment and the early-stopping rule as a principled account of the stability–plasticity trade-off (Dai et al., 23 Jul 2025).

The stated limitations are specific. Small-sample covariance estimation, especially on CUB, remains challenging because per-class sample sizes can be low. Performance depends on the quality of initial clustering, since poor clusters propagate noise to the initial novel distributions. The Gaussian class-conditional assumption may be violated in some domains, and diagonal covariance approximations degrade performance relative to full covariance. These limitations suggest that the method’s strongest performance should be expected when features are sufficiently well-behaved for full-covariance Gaussian modeling to be informative.

The future directions given in the paper remain within the same probabilistic design space: replacing k-means with deep clustering or adaptive mixture models, using hierarchical or empirical Bayes priors to stabilize covariance estimation under limited data, and integrating Bayesian nonparametrics such as Dirichlet processes for unknown class counts. Together, these directions indicate that VB-CGCD is best understood not as a fixed recipe, but as a Bayesian program for continual category discovery in which uncertainty over class geometry is central (Dai et al., 23 Jul 2025).

7. Acronym ambiguity and disambiguation

The shorthand “VB-CGCD” is not standardized across the broader variational Bayes literature. In the continual learning context, it denotes “Variational Bayes C-GCD,” the method described above for Continual Generalized Category Discovery (Dai et al., 23 Jul 2025).

In unrelated literatures, the same shorthand has been used only informally or retrospectively for different methods. In semi-blind graph signal recovery, it has been used to describe a Variational Bayes algorithm whose coordinate update for each adjacency weight yields a generalized compound confluent hypergeometric posterior; the corresponding paper itself studies “Statistical Graph Signal Recovery Using Variational Bayes” rather than a method formally named VB-CGCD (Torkamani et al., 2020). In variational logistic modeling, it has been used as an interpretive label for mean-field variational Bayes in conditionally conjugate exponential-family models solved by coordinate ascent updates, with the Jaakkola–Jordan bound linked formally to Polya–Gamma augmentation; again, the paper does not explicitly define the acronym (Durante et al., 2017). In Gaussian factor modeling, it has likewise been used as a convenience label for “Variational Bayes for Gaussian factor models under the cumulative shrinkage process,” not as the paper’s formal method name (Legramanti, 2020).

This nomenclatural overlap can create a superficial ambiguity. In current continual learning usage, however, VB-CGCD refers specifically to the covariance-aware Bayesian framework for continual generalized category discovery introduced in 2025, not to those unrelated variational Bayes constructions.

Topic to Video (Beta)

No one has generated a video about this topic yet.

Whiteboard

No one has generated a whiteboard explanation for this topic yet.

Follow Topic

Get notified by email when new papers are published related to Variational Bayes C-GCD (VB-CGCD).