Adaptive Discovering and Merging for Incremental Novel Class Discovery (2403.03382v1)
Abstract: One important desideratum of lifelong learning aims to discover novel classes from unlabelled data in a continuous manner. The central challenge is twofold: discovering and learning novel classes while mitigating the issue of catastrophic forgetting of established knowledge. To this end, we introduce a new paradigm called Adaptive Discovering and Merging (ADM) to discover novel categories adaptively in the incremental stage and integrate novel knowledge into the model without affecting the original knowledge. To discover novel classes adaptively, we decouple representation learning and novel class discovery, and use Triple Comparison (TC) and Probability Regularization (PR) to constrain the probability discrepancy and diversity for adaptive category assignment. To merge the learned novel knowledge adaptively, we propose a hybrid structure with base and novel branches named Adaptive Model Merging (AMM), which reduces the interference of the novel branch on the old classes to preserve the previous knowledge, and merges the novel branch to the base model without performance loss and parameter growth. Extensive experiments on several datasets show that ADM significantly outperforms existing class-incremental Novel Class Discovery (class-iNCD) approaches. Moreover, our AMM also benefits the class-incremental Learning (class-IL) task by alleviating the catastrophic forgetting problem.
- DO-Conv: Depthwise Over-parameterized Convolutional Layer. arXiv preprint arXiv:2006.12030.
- Adversarial Reciprocal Points Learning for Open Set Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence.
- Learning open set network with discriminative reciprocal points. In European Conference on Computer Vision, 507–522. Springer.
- A simple framework for contrastive learning of visual representations. In ICML.
- Reducing network agnostophobia. In Advances in Neural Information Processing Systems, 9157–9168.
- Acnet: Strengthening the kernel skeletons for powerful cnn via asymmetric convolution blocks. In Proceedings of the IEEE International Conference on Computer Vision, 1911–1920.
- Repvgg: Making vgg-style convnets great again. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 13733–13742.
- PODNet: Pooled Outputs Distillation for Small-Tasks Incremental Learning. In ECCV.
- Dytox: Transformers for continual learning with dynamic token expansion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9285–9295.
- A unified objective for novel class discovery. In ICCV.
- An empirical investigation of catastrophic forgetting in gradient-based neural networks. In ICLR.
- ExpandNets: Linear Over-parameterization to Train Compact Convolutional Networks. Advances in Neural Information Processing Systems, 33.
- Automatically discovering and learning new visual categories with ranking statistics. In ICLR.
- Learning to discover novel visual categories via deep transfer clustering. In ICCV.
- Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, 770–778.
- Distilling the Knowledge in a Neural Network. arXiv preprint arXiv:1503.02531.
- Learning to cluster in order to transfer across domains and tasks. In ICLR.
- Multi-class classification without multi-class labels. In ICLR.
- Joint representation learning and novel category discovery on single-and multi-modal data. In ICCV.
- Less-forgetting Learning in Deep Neural Networks. arXiv preprint arXiv:1607.00122.
- Class-Incremental Learning by Knowledge Distillation with Adaptive Feature Consolidation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 16071–16080.
- Learning multiple layers of features from tiny images.
- Kuhn, H. W. 1955. The Hungarian method for the assignment problem. Naval research logistics quarterly.
- Tiny imagenet visual recognition challenge. CS 231N.
- Learning without forgetting. TPAMI.
- Adaptive Aggregation Networks for Class-Incremental Learning. In CVPR.
- Residual tuning: Toward novel category discovery without labels. TNNLS.
- Catastrophic Interference in Connectionist Networks: The sequential Learning Problem. In Psychology of Learning and Motivation. Elsevier.
- icarl: Incremental classifier and representation learning. In CVPR.
- Class-incremental Novel Class Discovery. In European Conference on Computer Vision, 317–333. Springer.
- Progressive Neural Networks. arXiv preprint arXiv:1606.04671.
- Foster: Feature boosting and compression for class-incremental learning. In Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXV, 398–414. Springer.
- DER: Dynamically Expandable Representation for Class Incremental Learning. In CVPR.
- Diracnets: Training very deep neural networks without skip-connections. arXiv preprint arXiv:1706.00388.
- Neighborhood Contrastive Learning for Novel Class Discovery. In CVPR.
- Openmix: Reviving known knowledge for discovering novel visual categories in an open world. In CVPR.
- Self-sustaining representation expansion for non-exemplar class-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9296–9305.