PromptCCD: Learning Gaussian Mixture Prompt Pool for Continual Category Discovery (2407.19001v2)

Abstract: We tackle the problem of Continual Category Discovery (CCD), which aims to automatically discover novel categories in a continuous stream of unlabeled data while mitigating the challenge of catastrophic forgetting -- an open problem that persists even in conventional, fully supervised continual learning. To address this challenge, we propose PromptCCD, a simple yet effective framework that utilizes a Gaussian Mixture Model (GMM) as a prompting method for CCD. At the core of PromptCCD lies the Gaussian Mixture Prompting (GMP) module, which acts as a dynamic pool that updates over time to facilitate representation learning and prevent forgetting during category discovery. Moreover, GMP enables on-the-fly estimation of category numbers, allowing PromptCCD to discover categories in unlabeled data without prior knowledge of the category numbers. We extend the standard evaluation metric for Generalized Category Discovery (GCD) to CCD and benchmark state-of-the-art methods on diverse public datasets. PromptCCD significantly outperforms existing methods, demonstrating its effectiveness. Project page: https://visual-ai.github.io/promptccd .

Overview of PromptCCD: Continual Category Discovery Using Gaussian Mixture Prompts

Introduction

The paper presents PromptCCD, a novel framework targeting Continual Category Discovery (CCD) in machine learning. CCD addresses the challenge of discovering novel categories in a continuous stream of unlabelled data without succumbing to catastrophic forgetting, a well-known issue in continual learning. Leveraging the robustness of self-supervised vision foundation models like DINO, the authors propose the Gaussian Mixture Prompting (GMP) method to dynamically guide the learning process, enhance feature representations, and address the challenge of unknown category numbers.

Gaussian Mixture Prompting (GMP) Module

At the heart of PromptCCD lies the GMP module. GMP employs a Gaussian Mixture Model (GMM) to generate and manage prompts used for feature learning and category discovery. The process involves:

1. Dynamic Updates: The GMM dynamically updates over time, reflecting the evolving nature of unlabelled data streams in CCD.
2. Category Estimation: GMP allows on-the-fly estimation of category numbers, eliminating the need for prior knowledge on the exact number of categories in the dataset.
3. Dual Role Prompts: The prompts serve as task-specific guides and class prototypes, ensuring robust training and retention of previously learned categories.

Methodology

The framework begins by using a pre-trained DINO model for initial feature extraction. For subsequent stages, PromptCCD integrates the GMP module to fine-tune the model and dynamically adjust category estimations. The GMP module operates as follows:

• Feature Extraction: Extract features using the backbone model.
• Prompt Selection: Utilize the GMM to select the most relevant prompts dynamically.
• Training: Fine-tune the model using contrasting learning objectives, incorporating the selected prompts to guide and enhance learning.

This approach ensures that the model remains adaptable, effectively manages feature representations, and minimizes forgetting.

Results and Benchmarking

The efficacy of PromptCCD is validated across multiple datasets, including CIFAR100, ImageNet-100, and fine-grained datasets like CUB. The experimental results are summarized as follows:

• Superior Performance: PromptCCD consistently outperforms benchmark methods across various metrics, demonstrating improvement in 'All', 'Old', and 'New' accuracy measures.
• Scalability: The GMP module ensures that the model can handle growing categories, providing robust performance without degradation in new stages.
• Category Estimation: The ability to estimate the number of categories on-the-fly is highlighted as a unique strength, tackling one of the open challenges in CCD.

Implementation and Adaptations

The paper includes detailed implementation strategies for augmenting existing methods with the GMP module, validating the approach's flexibility and robustness. For instance:

• Enhanced Architectures: Integration of G{content}M with ViT showed improved performance, underlying the adaptability of the proposed design.
• Comparative Analysis: Consistent benchmarking with recent methods like PA-CGCD and MetaGCD under different evaluation protocols further solidifies the framework's superiority.

Implications and Future Directions

The implications of this research are multifaceted:

• Theoretical Impact: Introducing a dynamic prompt-based approach rooted in GMM offers a novel perspective to address continual learning.
• Practical Applications: In real-world scenarios where data streams are continuous and unlabelled, PromptCCD provides a scalable solution for discovering novel categories while retaining past knowledge.

Future developments may explore the integration of more sophisticated self-supervised models and further optimization of the GMM-based prompt generation to enhance the scalability and efficacy of PromptCCD. Additionally, addressing potential biases in data and ensuring robustness against error accumulation over longer sequences remain areas for improvement.

Conclusion

PromptCCD's innovative use of Gaussian Mixture Prompts establishes a new benchmark in Continual Category Discovery. By dynamically adjusting to new data, estimating category numbers on-the-fly, and mitigating catastrophic forgetting, the framework significantly advances the state-of-the-art in continuous learning environments. The extensive experiments and strategic improvements underscore its potential for broad application across various machine learning domains.