Class Similarity Transition: Decoupling Class Similarities and Imbalance from Generalized Few-shot Segmentation (2404.05111v1)
Abstract: In Generalized Few-shot Segmentation (GFSS), a model is trained with a large corpus of base class samples and then adapted on limited samples of novel classes. This paper focuses on the relevance between base and novel classes, and improves GFSS in two aspects: 1) mining the similarity between base and novel classes to promote the learning of novel classes, and 2) mitigating the class imbalance issue caused by the volume difference between the support set and the training set. Specifically, we first propose a similarity transition matrix to guide the learning of novel classes with base class knowledge. Then, we leverage the Label-Distribution-Aware Margin (LDAM) loss and Transductive Inference to the GFSS task to address the problem of class imbalance as well as overfitting the support set. In addition, by extending the probability transition matrix, the proposed method can mitigate the catastrophic forgetting of base classes when learning novel classes. With a simple training phase, our proposed method can be applied to any segmentation network trained on base classes. We validated our methods on the adapted version of OpenEarthMap. Compared to existing GFSS baselines, our method excels them all from 3% to 7% and ranks second in the OpenEarthMap Land Cover Mapping Few-Shot Challenge at the completion of this paper. Code: https://github.com/earth-insights/ClassTrans
- Few-shot segmentation without meta-learning: A good transductive inference is all you need?, 2021a.
- Few-shot segmentation without meta-learning: A good transductive inference is all you need?, 2021b.
- Learning imbalanced datasets with label-distribution-aware margin loss, 2019.
- An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.
- Unsupervised domain adaptation by backpropagation, 2015.
- Domain-adversarial training of neural networks, 2016.
- Rich feature hierarchies for accurate object detection and semantic segmentation, 2014.
- A strong baseline for generalized few-shot semantic segmentation, 2023.
- Deep residual learning for image recognition, 2015.
- Adjusting decision boundary for class imbalanced learning, 2020.
- Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems. Curran Associates, Inc., 2012.
- Learning what not to segment: A new perspective on few-shot segmentation, 2022.
- Microsoft coco: Common objects in context, 2015.
- Learning orthogonal prototypes for generalized few-shot semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11319–11328, 2023a.
- Harmonizing base and novel classes: A class-contrastive approach for generalized few-shot segmentation, 2023b.
- A convnet for the 2020s. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11976–11986, 2022.
- Long-tail learning via logit adjustment, 2021.
- U-net: Convolutional networks for biomedical image segmentation, 2015.
- Prototypical networks for few-shot learning, 2017.
- Martin Thoma. A survey of semantic segmentation, 2016.
- Posterior re-calibration for imbalanced datasets, 2020a.
- Prior guided feature enrichment network for few-shot segmentation, 2020b.
- Generalized few-shot semantic segmentation, 2022.
- Panet: Few-shot image semantic segmentation with prototype alignment, 2020.
- Label hierarchy transition: Delving into class hierarchies to enhance deep classifiers, 2023.
- Long-tailed recognition by routing diverse distribution-aware experts, 2022.
- Openearthmap: A benchmark dataset for global high-resolution land cover mapping, 2022.
- Learning from multiple experts: Self-paced knowledge distillation for long-tailed classification, 2020.
- Unified perceptual parsing for scene understanding. In Proceedings of the European conference on computer vision (ECCV), pages 418–434, 2018.
- Identifying and compensating for feature deviation in imbalanced deep learning, 2022.
- Distribution alignment: A unified framework for long-tail visual recognition, 2021.
- Deep long-tailed learning: A survey, 2023.
- Pyramid scene parsing network. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2881–2890, 2017.
- Exploiting associations between word clusters and document classes for cross-domain text categorization. Statistical Analysis and Data Mining, 4:100 – 114, 2011.
- Triplex transfer learning: Exploiting both shared and distinct concepts for text classification. IEEE Transactions on Cybernetics, 44(7):1191–1203, 2014.
- A comprehensive survey on transfer learning, 2020.
- Shihong Wang (10 papers)
- Ruixun Liu (5 papers)
- Kaiyu Li (17 papers)
- Jiawei Jiang (47 papers)
- Xiangyong Cao (50 papers)