Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Towards Non-Exemplar Semi-Supervised Class-Incremental Learning (2403.18291v1)

Published 27 Mar 2024 in cs.CV

Abstract: Deep neural networks perform remarkably well in close-world scenarios. However, novel classes emerged continually in real applications, making it necessary to learn incrementally. Class-incremental learning (CIL) aims to gradually recognize new classes while maintaining the discriminability of old ones. Existing CIL methods have two limitations: a heavy reliance on preserving old data for forgetting mitigation and the need for vast labeled data for knowledge adaptation. To overcome these issues, we propose a non-exemplar semi-supervised CIL framework with contrastive learning and semi-supervised incremental prototype classifier (Semi-IPC). On the one hand, contrastive learning helps the model learn rich representations, easing the trade-off between learning representations of new classes and forgetting that of old classes. On the other hand, Semi-IPC learns a prototype for each class with unsupervised regularization, enabling the model to incrementally learn from partially labeled new data while maintaining the knowledge of old classes. Experiments on benchmark datasets demonstrate the strong performance of our method: without storing any old samples and only using less than 1% of labels, Semi-IPC outperforms advanced exemplar-based methods. We hope our work offers new insights for future CIL research. The code will be made publicly available.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (100)
  1. John T Wixted. The psychology and neuroscience of forgetting. Annual Review of Psychology, 55:235–269, 2004.
  2. Endel Tulving. Episodic and semantic memory. Organization of Memory, pages 381–403, 1972.
  3. A continual learning survey: Defying forgetting in classification tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021.
  4. An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211, 2013.
  5. M. McCloskey and N. J. Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. Psychology of Learning and Motivation, 24:109–165, 1989.
  6. Class-incremental continual learning into the extended der-verse. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
  7. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, 114(13):3521–3526, 2017.
  8. Memory aware synapses: Learning what (not) to forget. In Proceedings of the European Conference on Computer Vision, pages 139–154, 2018.
  9. Learning without forgetting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(12):2935–2947, 2017.
  10. Learning without memorizing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5138–5146, 2019.
  11. Deesil: Deep-shallow incremental learning. In Proceedings of the European Conference on Computer Vision Workshops, pages 0–0, 2018.
  12. Il2m: Class incremental learning with dual memory. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 583–592, 2019.
  13. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2001–2010, 2017.
  14. End-to-end incremental learning. In Proceedings of the European Conference on Computer Vision, pages 233–248, 2018.
  15. Progressive neural networks. arXiv preprint arXiv:1606.04671, 2016.
  16. Packnet: Adding multiple tasks to a single network by iterative pruning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 7765–7773, 2018.
  17. Overcoming catastrophic forgetting with hard attention to the task. In International Conference on Machine Learning, pages 4548–4557. PMLR, 2018.
  18. Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547, 2017.
  19. Learning a unified classifier incrementally via rebalancing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 831–839, 2019.
  20. Podnet: Pooled outputs distillation for small-tasks incremental learning. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pages 86–102. Springer, 2020.
  21. Continual lifelong learning with neural networks: A review. Neural Networks, 113:54–71, 2019.
  22. Federated learning: Challenges, methods, and future directions. IEEE Signal Processing Magazine, 37(3):50–60, 2020.
  23. Information security and privacy in healthcare: current state of research. International Journal of Internet and Enterprise Management, 6(4):279–314, 2010.
  24. Engrams and circuits crucial for systems consolidation of a memory. Science, 356(6333):73–78, 2017.
  25. What learning systems do intelligent agents need? complementary learning systems theory updated. Trends in Cognitive Sciences, 20(7):512–534, 2016.
  26. Continual learning with deep generative replay. In Advances in Neural Information Processing Systems, pages 2990–2999, 2017.
  27. Memory replay gans: Learning to generate new categories without forgetting. In Advances in Neural Information Processing Systems, 2018.
  28. Fearnet: Brain-inspired model for incremental learning. In International Conference on Learning Representations, 2018.
  29. Few-shot incremental learning with continually evolved classifiers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12455–12464, 2021.
  30. Convolutional prototype network for open set recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(5):2358–2370, 2020.
  31. Prototype augmentation and self-supervision for incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5871–5880, 2021.
  32. Continual learning through synaptic intelligence. In International Conference on Machine Learning, pages 3987–3995. PMLR, 2017.
  33. More classifiers, less forgetting: A generic multi-classifier paradigm for incremental learning. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXVI 16, pages 699–716. Springer, 2020.
  34. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015.
  35. Deep generative dual memory network for continual learning. arXiv preprint arXiv:1710.10368, 2017.
  36. Continual learning with deep generative replay. arXiv preprint arXiv:1705.08690, 2017.
  37. Class incremental learning: A review and performance evaluation. Acta Automatica Sinica, 49(3):635–660, 2023.
  38. Dynamic support network for few-shot class incremental learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(3):2945–2951, 2022.
  39. Der: Dynamically expandable representation for class incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3014–3023, 2021.
  40. Foster: Feature boosting and compression for class-incremental learning. In Proceedings of the European Conference on Computer Vision, 2022.
  41. Dytox: Transformers for continual learning with dynamic token expansion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9285–9295, 2022.
  42. Dense network expansion for class incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11858–11867, 2023.
  43. Learning to prompt for continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 139–149, 2022.
  44. Dualprompt: Complementary prompting for rehearsal-free continual learning. In Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXVI, pages 631–648. Springer, 2022.
  45. Coda-prompt: Continual decomposed attention-based prompting for rehearsal-free continual learning. arXiv preprint arXiv:2211.13218, 2022.
  46. S-prompts learning with pre-trained transformers: An occam’s razor for domain incremental learning. arXiv preprint arXiv:2207.12819, 2022.
  47. A theoretical study on solving continual learning. In Advances in Neural Information Processing Systems, 2022.
  48. Few-shot class-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12183–12192, 2020.
  49. Self-promoted prototype refinement for few-shot class-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6801–6810, 2021.
  50. Few-shot class-incremental learning via relation knowledge distillation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 1255–1263, 2021.
  51. Uncertainty-guided semi-supervised few-shot class-incremental learning with knowledge distillation. IEEE Transactions on Multimedia, 2022.
  52. Uncertainty-aware distillation for semi-supervised few-shot class-incremental learning. IEEE Transactions on Neural Networks and Learning Systems, 2023.
  53. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018.
  54. A simple framework for contrastive learning of visual representations. In International Conference on Machine Learning, pages 1597–1607. PMLR, 2020.
  55. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9729–9738, 2020.
  56. Bootstrap your own latent: A new approach to self-supervised learning. arXiv preprint arXiv:2006.07733, 2020.
  57. Exploring simple siamese representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15750–15758, 2021.
  58. Unsupervised learning of visual features by contrasting cluster assignments. Advances in neural information processing systems, 33:9912–9924, 2020.
  59. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9650–9660, 2021.
  60. Barlow twins: Self-supervised learning via redundancy reduction. In International Conference on Machine Learning, pages 12310–12320. PMLR, 2021.
  61. Vicreg: Variance-invariance-covariance regularization for self-supervised learning. In ICLR 2022-10th International Conference on Learning Representations, 2022.
  62. Rahaf Aljundi. Continual learning in neural networks. ArXiv, abs/1910.02718, 2019.
  63. Intrinsic dimension of data representations in deep neural networks. In Advances in Neural Information Processing Systems, 2019.
  64. Learning diverse and discriminative representations via the principle of maximal coding rate reduction. Advances in Neural Information Processing Systems, 33, 2020.
  65. Understanding the role of self-supervised learning in out-of-distribution detection task. arXiv preprint arXiv:2110.13435, 2021.
  66. Karl Pearson. Liii. on lines and planes of closest fit to systems of points in space. Philosophical Magazine and Journal of Science, 2(11):559–572, 1901.
  67. Learning multiple layers of features from tiny images. 2009.
  68. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 770–778, 2016.
  69. Revisiting training strategies and generalization performance in deep metric learning. In International Conference on Machine Learning, 2020.
  70. Unsupervised data augmentation for consistency training. Advances in Neural Information Processing Systems, 33:6256–6268, 2020.
  71. Remixmatch: Semi-supervised learning with distribution alignment and augmentation anchoring. arXiv preprint arXiv:1911.09785, 2019.
  72. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. Advances in Neural Information Processing Systems, 33:596–608, 2020.
  73. Randaugment: Practical automated data augmentation with a reduced search space. Advances in Neural Information Processing Systems, 33:18613–18624, 2020.
  74. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255. Ieee, 2009.
  75. Matching networks for one shot learning. Advances in Neural Information Processing Systems, 29, 2016.
  76. Efficient lifelong learning with a-gem. In International Conference on Learning Representations, 2019.
  77. Ss-il: Separated softmax for incremental learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 844–853, 2021.
  78. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5533–5542, 2017.
  79. Podnet: Pooled outputs distillation for small-tasks incremental learning. In Proceedings of the European Conference on Computer Vision, pages 86–102, 2020.
  80. A model or 603 exemplars: Towards memory-efficient class-incremental learning. In International Conference on Learning Representations, 2023.
  81. Self-sustaining representation expansion for non-exemplar class-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9296–9305, 2022.
  82. Fetril: Feature translation for exemplar-free class-incremental learning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 3911–3920, January 2023.
  83. On learning the geodesic path for incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1591–1600, 2021.
  84. Mnemonics training: Multi-class incremental learning without forgetting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12245–12254, 2020.
  85. Adaptive aggregation networks for class-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2544–2553, 2021.
  86. Mimicking the oracle: an initial phase decorrelation approach for class incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16722–16731, 2022.
  87. Learning to imagine: Diversify memory for incremental learning using unlabeled data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9549–9558, 2022.
  88. Class-incremental learning by knowledge distillation with adaptive feature consolidation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16071–16080, 2022.
  89. Dualnet: Continual learning, fast and slow. Advances in Neural Information Processing Systems, 34, 2021.
  90. Bilateral memory consolidation for continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16026–16035, 2023.
  91. Optimizing mode connectivity for class incremental learning. In International Conference on Machine Learning, 2023.
  92. Forward compatible few-shot class-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9046–9056, 2022.
  93. Incremental few-shot learning via vector quantization in deep embedded space. In International Conference on Learning Representations, 2020.
  94. Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983, 2016.
  95. Semi-supervised few-shot class-incremental learning. In 2021 IEEE International Conference on Image Processing, pages 1239–1243. IEEE, 2021.
  96. Imitating the oracle: Towards calibrated model for class incremental learning. Neural Networks, 164:38–48, 2023.
  97. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2021.
  98. Mixmatch: A holistic approach to semi-supervised learning. Advances in Neural Information Processing Systems, 32, 2019.
  99. Self-supervised models are continual learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9621–9630, 2022.
  100. Continual barlow twins: continual self-supervised learning for remote sensing semantic segmentation. arXiv preprint arXiv:2205.11319, 2022.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Wenzhuo Liu (18 papers)
  2. Fei Zhu (49 papers)
  3. Cheng-Lin Liu (71 papers)
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets