Papers
Topics
Authors
Recent
2000 character limit reached

Auto-Generating Weak Labels for Real & Synthetic Data to Improve Label-Scarce Medical Image Segmentation (2404.17033v1)

Published 25 Apr 2024 in cs.CV

Abstract: The high cost of creating pixel-by-pixel gold-standard labels, limited expert availability, and presence of diverse tasks make it challenging to generate segmentation labels to train deep learning models for medical imaging tasks. In this work, we present a new approach to overcome the hurdle of costly medical image labeling by leveraging foundation models like Segment Anything Model (SAM) and its medical alternate MedSAM. Our pipeline has the ability to generate weak labels for any unlabeled medical image and subsequently use it to augment label-scarce datasets. We perform this by leveraging a model trained on a few gold-standard labels and using it to intelligently prompt MedSAM for weak label generation. This automation eliminates the manual prompting step in MedSAM, creating a streamlined process for generating labels for both real and synthetic images, regardless of quantity. We conduct experiments on label-scarce settings for multiple tasks pertaining to modalities ranging from ultrasound, dermatology, and X-rays to demonstrate the usefulness of our pipeline. The code is available at https://github.com/stanfordmlgroup/Auto-Generate-WLs/.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (37)
  1. Dataset of breast ultrasound images. Data in Brief, 28:104863, 2020. ISSN 2352-3409. https://doi.org/10.1016/j.dib.2019.104863. URL https://www.sciencedirect.com/science/article/pii/S2352340919312181.
  2. One-shot localization and segmentation of medical images with foundation models, 2023.
  3. Universeg: Universal medical image segmentation. International Conference on Computer Vision, 2023.
  4. Ma-sam: Modality-agnostic sam adaptation for 3d medical image segmentation, 2023.
  5. Transunet: Transformers make strong encoders for medical image segmentation, 2021.
  6. All-in-sam: from weak annotation to pixel-wise nuclei segmentation with prompt-based finetuning, 2023.
  7. Zero-shot performance of the segment anything model (sam) in 2d medical imaging: A comprehensive evaluation and practical guidelines. ArXiv, abs/2305.00109, 2023. URL https://api.semanticscholar.org/CorpusID:258426532.
  8. Curation of the candid-ptx dataset with free-text reports. Radiology. Artificial intelligence, 3 6:e210136, 2021. URL https://api.semanticscholar.org/CorpusID:243370709.
  9. A dataset of covid-19 x-ray chest images. Data in Brief, 47:109000, 2023.
  10. An ensemble classification-based approach applied to retinal blood vessel segmentation. IEEE Transactions on Biomedical Engineering, 59(9):2538–2548, 2012. 10.1109/TBME.2012.2205687.
  11. Generative adversarial networks. Communications of the ACM, 63(11):139–144, 2020.
  12. Skin lesion analysis toward melanoma detection: A challenge at the international symposium on biomedical imaging (isbi) 2016, hosted by the international skin imaging collaboration (isic). ArXiv, abs/1605.01397, 2016. URL https://api.semanticscholar.org/CorpusID:260439738.
  13. Computer-vision benchmark segment-anything model (sam) in medical images: Accuracy in 12 datasets, 2023.
  14. Denoising diffusion probabilistic models, 2020.
  15. Diffusion models in medical imaging: A comprehensive survey. Medical Image Analysis, 88:102846, 2023. ISSN 1361-8415. https://doi.org/10.1016/j.media.2023.102846. URL https://www.sciencedirect.com/science/article/pii/S1361841523001068.
  16. Segment anything. arXiv:2304.02643, 2023.
  17. Medlsam: Localize and segment anything model for 3d ct images, 2023.
  18. Samus: Adapting segment anything model for clinically-friendly and generalizable ultrasound image segmentation, 2023.
  19. Segment anything in medical images. arXiv preprint arXiv:2304.12306, 2023.
  20. U-mamba: Enhancing long-range dependency for biomedical image segmentation. arXiv preprint arXiv:2401.04722, 2024.
  21. Zero-shot segmentation of eye features using the segment anything model (sam). ArXiv, abs/2311.08077, 2023. URL https://api.semanticscholar.org/CorpusID:265158106.
  22. V-net: Fully convolutional neural networks for volumetric medical image segmentation. In 2016 Fourth International Conference on 3D Vision (3DV), pages 565–571, 2016. 10.1109/3DV.2016.79.
  23. Comprehensive multimodal segmentation in medical imaging: Combining yolov8 with sam and hq-sam models. In 2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), pages 2584–2590, Los Alamitos, CA, USA, oct 2023. IEEE Computer Society. 10.1109/ICCVW60793.2023.00273. URL https://doi.ieeecomputersociety.org/10.1109/ICCVW60793.2023.00273.
  24. Empirical evaluation of the segment anything model (sam) for brain tumor segmentation. 2023. URL https://api.semanticscholar.org/CorpusID:263829153.
  25. Self-guided few-shot semantic segmentation for remote sensing imagery based on large vision models, 2023.
  26. U-net: Convolutional networks for biomedical image segmentation. In Nassir Navab, Joachim Hornegger, William M. Wells, and Alejandro F. Frangi, editors, Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, pages 234–241, Cham, 2015. Springer International Publishing. ISBN 978-3-319-24574-4.
  27. Autosam: Adapting sam to medical images by overloading the prompt encoder, 2023.
  28. Cemb-sam: Segment anything model with condition embedding for joint learning from heterogeneous datasets. In M. Emre Celebi, Md Sirajus Salekin, Hyunwoo Kim, Shadi Albarqouni, Catarina Barata, Allan Halpern, Philipp Tschandl, Marc Combalia, Yuan Liu, Ghada Zamzmi, Joshua Levy, Huzefa Rangwala, Annika Reinke, Diya Wynn, Bennett Landman, Won-Ki Jeong, Yiqing Shen, Zhongying Deng, Spyridon Bakas, Xiaoxiao Li, Chen Qin, Nicola Rieke, Holger Roth, and Daguang Xu, editors, Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 Workshops, pages 275–284, Cham, 2023. Springer Nature Switzerland. ISBN 978-3-031-47401-9.
  29. Self-supervised pre-training of swin transformers for 3d medical image analysis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20730–20740, 2022.
  30. KiU-Net: Towards Accurate Segmentation of Biomedical Images Using Over-Complete Representations, pages 363–373. 09 2020. ISBN 978-3-030-59718-4. 10.1007/978-3-030-59719-1_36.
  31. Medical transformer: Gated axial-attention for medical image segmentation. In Medical Image Computing and Computer Assisted Intervention – MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part I, page 36–46, Berlin, Heidelberg, 2021. Springer-Verlag. ISBN 978-3-030-87192-5. 10.1007/978-3-030-87193-2_4. URL https://doi.org/10.1007/978-3-030-87193-2_4.
  32. Caption anything: Interactive image description with diverse multimodal controls, 2023.
  33. Diffusion models for implicit image segmentation ensembles, 2021.
  34. Self-prompting large vision models for few-shot medical image segmentation. In Lisa Koch, M. Jorge Cardoso, Enzo Ferrante, Konstantinos Kamnitsas, Mobarakol Islam, Meirui Jiang, Nicola Rieke, Sotirios A. Tsaftaris, and Dong Yang, editors, Domain Adaptation and Representation Transfer, pages 156–167, Cham, 2024. Springer Nature Switzerland. ISBN 978-3-031-45857-6.
  35. Inpaint anything: Segment anything meets image inpainting, 2023.
  36. Input augmentation with sam: Boosting medical image segmentation with segmentation foundation model. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 129–139. Springer, 2023.
  37. Unet++: A nested u-net architecture for medical image segmentation. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support : 4th International Workshop, DLMIA 2018, and 8th International Workshop, ML-CDS 2018, held in conjunction with MICCAI 2018, Granada, Spain, S…, 11045:3–11, 2018. URL https://api.semanticscholar.org/CorpusID:50786304.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now

Whiteboard

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Tweets

Sign up for free to view the 1 tweet with 0 likes about this paper.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up for Free