Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Image-Object-Specific Prompt Learning for Few-Shot Class-Incremental Learning (2309.02833v2)

Published 6 Sep 2023 in cs.CV

Abstract: While many FSCIL studies have been undertaken, achieving satisfactory performance, especially during incremental sessions, has remained challenging. One prominent challenge is that the encoder, trained with an ample base session training set, often underperforms in incremental sessions. In this study, we introduce a novel training framework for FSCIL, capitalizing on the generalizability of the Contrastive Language-Image Pre-training (CLIP) model to unseen classes. We achieve this by formulating image-object-specific (IOS) classifiers for the input images. Here, an IOS classifier refers to one that targets specific attributes (like wings or wheels) of class objects rather than the image's background. To create these IOS classifiers, we encode a bias prompt into the classifiers using our specially designed module, which harnesses key-prompt pairs to pinpoint the IOS features of classes in each session. From an FSCIL standpoint, our framework is structured to retain previous knowledge and swiftly adapt to new sessions without forgetting or overfitting. This considers the updatability of modules in each session and some tricks empirically found for fast convergence. Our approach consistently demonstrates superior performance compared to state-of-the-art methods across the miniImageNet, CIFAR100, and CUB200 datasets. Further, we provide additional experiments to validate our learned model's ability to achieve IOS classifiers. We also conduct ablation studies to analyze the impact of each module within the architecture.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (34)
  1. Generic attention-model explainability for interpreting bi-modal and encoder-decoder transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 397–406, 2021.
  2. Incremental few-shot learning via vector quantization in deep embedded space. In International Conference on Learning Representations, 2020.
  3. Semantic-aware knowledge distillation for few-shot class-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2534–2543, 2021.
  4. Multimodal parameter-efficient few-shot class incremental learning. arXiv preprint arXiv:2303.04751, 2023.
  5. Few-shot class-incremental learning via relation knowledge distillation. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 1255–1263, 2021.
  6. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.
  7. Contextual prompt learning for vision-language understanding. arXiv preprint arXiv:2307.00910, 2023.
  8. Constrained few-shot class-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9057–9067, 2022.
  9. Memorizing complementation network for few-shot class-incremental learning. arXiv preprint arXiv:2208.05610, 2022.
  10. Introducing language guidance in prompt-based continual learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11463–11473, 2023.
  11. Cifar-10, cifar-100 (canadian institute for advanced research), 2019.
  12. Multi-view class incremental learning. arXiv preprint arXiv:2306.09675, 2023.
  13. Few-shot class-incremental learning via entropy-regularized data-free replay. arXiv preprint arXiv:2207.11213, 2022.
  14. Mudpt: Multi-modal deep-symphysis prompt tuning for large pre-trained vision-language models. arXiv preprint arXiv:2306.11400, 2023.
  15. Few-shot class-incremental learning from an open-set perspective. In European Conference on Computer Vision, pages 382–397. Springer, 2022.
  16. Semantic-visual guided transformer for few-shot class-incremental learning. arXiv preprint arXiv:2303.15494, 2023.
  17. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, pages 8748–8763. PMLR, 2021.
  18. Sebastian Ruder. An overview of gradient descent optimization algorithms. arXiv preprint arXiv:1609.04747, 2016.
  19. Imagenet large scale visual recognition challenge. International journal of computer vision, 115(3):211–252, 2015.
  20. Learning with fantasy: Semantic-aware virtual contrastive constraint for few-shot class-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24183–24192, 2023.
  21. When prompt-based incremental learning does not meet strong pretraining. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1706–1716, 2023.
  22. Few-shot class-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12183–12192, 2020.
  23. Matching networks for one shot learning. Advances in neural information processing systems, 29:3630–3638, 2016.
  24. The caltech-ucsd birds-200-2011 dataset. 2011.
  25. Attriclip: A non-incremental learner for incremental knowledge learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3654–3663, 2023.
  26. Der: Dynamically expandable representation for class incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3014–3023, 2021.
  27. Balanced supervised contrastive learning for few-shot class-incremental learning. arXiv preprint arXiv:2305.16687, 2023.
  28. Few-shot incremental learning with continually evolved classifiers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12455–12464, 2021.
  29. Forward compatible few-shot class-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9046–9056, 2022a.
  30. Few-shot class-incremental learning by sampling multi-phase tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022b.
  31. Learning without forgetting for vision-language models. arXiv preprint arXiv:2305.19270, 2023.
  32. Conditional prompt learning for vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16816–16825, 2022c.
  33. Learning to prompt for vision-language models. International Journal of Computer Vision, 130(9):2337–2348, 2022d.
  34. Margin-based few-shot class-incremental learning with class-level overfitting mitigation. arXiv preprint arXiv:2210.04524, 2022.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. In-Ug Yoon (3 papers)
  2. Tae-Min Choi (5 papers)
  3. Sun-Kyung Lee (4 papers)
  4. Young-Min Kim (37 papers)
  5. Jong-Hwan Kim (25 papers)
Citations (5)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now