Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Few Shot Class Incremental Learning using Vision-Language models (2405.01040v2)

Published 2 May 2024 in cs.CV, cs.CL, and eess.IV

Abstract: Recent advancements in deep learning have demonstrated remarkable performance comparable to human capabilities across various supervised computer vision tasks. However, the prevalent assumption of having an extensive pool of training data encompassing all classes prior to model training often diverges from real-world scenarios, where limited data availability for novel classes is the norm. The challenge emerges in seamlessly integrating new classes with few samples into the training data, demanding the model to adeptly accommodate these additions without compromising its performance on base classes. To address this exigency, the research community has introduced several solutions under the realm of few-shot class incremental learning (FSCIL). In this study, we introduce an innovative FSCIL framework that utilizes language regularizer and subspace regularizer. During base training, the language regularizer helps incorporate semantic information extracted from a Vision-LLM. The subspace regularizer helps in facilitating the model's acquisition of nuanced connections between image and text semantics inherent to base classes during incremental training. Our proposed framework not only empowers the model to embrace novel classes with limited data, but also ensures the preservation of performance on base classes. To substantiate the efficacy of our approach, we conduct comprehensive experiments on three distinct FSCIL benchmarks, where our framework attains state-of-the-art performance.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (38)
  1. Semantics-driven generative replay for few-shot class incremental learning. In Proceedings of the 30th ACM International Conference on Multimedia (pp. 5246–5254).
  2. Label-embedding for attribute-based classification. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 819–826).
  3. Subspace regularizers for few-shot class incremental learning. In International Conference on Learning Representations.
  4. Convex multi-task learning by clustering. In Artificial Intelligence and Statistics (pp. 65–73). PMLR.
  5. Il2m: Class incremental learning with dual memory. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 583–592).
  6. Enriching word vectors with subword information. Transactions of the association for computational linguistics, 5, 135–146.
  7. End-to-end incremental learning. In Proceedings of the European conference on computer vision (ECCV) (pp. 233–248).
  8. Importance of semantic representation: Dataless classification. In Aaai (pp. 830–835). volume 2.
  9. Incremental few-shot learning via vector quantization in deep embedded space. In International Conference on Learning Representations.
  10. Semantic-aware knowledge distillation for few-shot class-incremental learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 2534–2543).
  11. Metafscil: A meta-learning approach for few-shot class incremental learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 14166–14175).
  12. Dynamic few-shot visual learning without forgetting. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4367–4375).
  13. Zero-shot learning using graph regularized latent discriminative cross-domain triplets. In Proceedings of the 11th Indian Conference on Computer Vision, Graphics and Image Processing (pp. 1–9).
  14. Learning a unified classifier incrementally via rebalancing. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 831–839).
  15. Zero-data learning of new tasks. In AAAI (p. 3). volume 1.
  16. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In International Conference on Machine Learning (pp. 12888–12900). PMLR.
  17. Scaling language-image pre-training via masking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 23390–23400).
  18. Few-shot class-incremental learning via entropy-regularized data-free replay. In European Conference on Computer Vision (pp. 146–162). Springer.
  19. Generative feature replay for class-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (pp. 226–227).
  20. Class-incremental learning: survey and performance evaluation on image classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45, 5513–5533.
  21. Few-shot lifelong learning. In Proceedings of the AAAI Conference on Artificial Intelligence (pp. 2337–2345). volume 35.
  22. GloVe: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 1532–1543). Doha, Qatar: Association for Computational Linguistics.
  23. A review of generalized zero-shot learning methods. IEEE transactions on pattern analysis and machine intelligence, .
  24. Low-shot learning with imprinted weights. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 5822–5830).
  25. Learning transferable visual models from natural language supervision. In International conference on machine learning (pp. 8748–8763). PMLR.
  26. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition (pp. 2001–2010).
  27. Incremental few-shot learning with attention attractor networks. Advances in neural information processing systems, 32.
  28. Baby steps towards few-shot learning with multiple semantics. Pattern Recognition Letters, 160, 142–147.
  29. Overcoming catastrophic forgetting in incremental few-shot learning by finding flat minima. Advances in neural information processing systems, 34, 6747–6761.
  30. Few-shot class-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 12183–12192).
  31. Class-incremental learning with generative classifiers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 3611–3620).
  32. Learnable expansion-and-compression network for few-shot class-incremental learning. arXiv preprint arXiv:2104.02281, .
  33. Xtarnet: Learning to extract task-adaptive representation for incremental few-shot learning. In International Conference on Machine Learning (pp. 10852–10860). PMLR.
  34. Few-shot incremental learning with continually evolved classifiers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 12455–12464).
  35. A simple framework for open-vocabulary segmentation and detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 1020–1031).
  36. Deep class-incremental learning: A survey. arXiv preprint arXiv:2302.03648, .
  37. Class-incremental learning via dual augmentation. Advances in Neural Information Processing Systems, 34, 14306–14318.
  38. Self-promoted prototype refinement for few-shot class-incremental learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 6801–6810).

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now