DINOv2 based Self Supervised Learning For Few Shot Medical Image Segmentation (2403.03273v1)
Abstract: Deep learning models have emerged as the cornerstone of medical image segmentation, but their efficacy hinges on the availability of extensive manually labeled datasets and their adaptability to unforeseen categories remains a challenge. Few-shot segmentation (FSS) offers a promising solution by endowing models with the capacity to learn novel classes from limited labeled examples. A leading method for FSS is ALPNet, which compares features between the query image and the few available support segmented images. A key question about using ALPNet is how to design its features. In this work, we delve into the potential of using features from DINOv2, which is a foundational self-supervised learning model in computer vision. Leveraging the strengths of ALPNet and harnessing the feature extraction capabilities of DINOv2, we present a novel approach to few-shot segmentation that not only enhances performance but also paves the way for more robust and adaptable medical image analysis.
- “Generalizing from a few examples: A survey on few-shot learning,” ACM Comput. Surv., vol. 53, no. 3, 2020.
- “Prototypical networks for few-shot learning,” Advances in neural information processing systems, vol. 30, 2017.
- “Self-supervision with superpixels: Training few-shot medical image segmentation without annotation,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXIX 16. Springer, 2020, pp. 762–780.
- “Dinov2: Learning robust visual features without supervision,” arXiv:2304.07193, 2023.
- “Cross-reference transformer for few-shot medical image segmentation,” arXiv preprint arXiv:2304.09630, 2023.
- “Few-shot medical image segmentation with cycle-resemblance attention,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2023, pp. 2488–2497.
- “A survey of self-supervised learning from multiple perspectives: Algorithms, theory, applications and future trends,” arXiv preprint arXiv:2301.05712, 2023.
- “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint arXiv:2010.11929, 2020.
- “Efficient graph-based image segmentation,” International journal of computer vision, vol. 59, pp. 167–181, 2004.
- “Panet: Few-shot image semantic segmentation with prototype alignment,” in proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 9197–9206.
- “Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs,” IEEE transactions on pattern analysis and machine intelligence, vol. 40, no. 4, pp. 834–848, 2017.
- “Miccai multi-atlas labeling beyond the cranial vault–workshop and challenge,” in Proc. MICCAI Multi-Atlas Labeling Beyond Cranial Vault—Workshop Challenge, 2015, vol. 5, p. 12.
- A. Emre Kavur et al., “CHAOS challenge - combined (CT-MR) healthy abdominal organ segmentation,” Medical Image Analysis, vol. 69, pp. 101950, apr 2021.
- “‘squeeze & excite’guided few-shot segmentation of volumetric images,” Medical image analysis, vol. 59, pp. 101587, 2020.
- “Lora: Low-rank adaptation of large language models,” arXiv preprint arXiv:2106.09685, 2021.
- “Masked-attention mask transformer for universal image segmentation,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 1290–1299.