Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
194 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

ScribblePrompt: Fast and Flexible Interactive Segmentation for Any Biomedical Image (2312.07381v3)

Published 12 Dec 2023 in cs.CV and eess.IV

Abstract: Biomedical image segmentation is a crucial part of both scientific research and clinical care. With enough labelled data, deep learning models can be trained to accurately automate specific biomedical image segmentation tasks. However, manually segmenting images to create training data is highly labor intensive and requires domain expertise. We present \emph{ScribblePrompt}, a flexible neural network based interactive segmentation tool for biomedical imaging that enables human annotators to segment previously unseen structures using scribbles, clicks, and bounding boxes. Through rigorous quantitative experiments, we demonstrate that given comparable amounts of interaction, ScribblePrompt produces more accurate segmentations than previous methods on datasets unseen during training. In a user study with domain experts, ScribblePrompt reduced annotation time by 28% while improving Dice by 15% compared to the next best method. ScribblePrompt's success rests on a set of careful design decisions. These include a training strategy that incorporates both a highly diverse set of images and tasks, novel algorithms for simulated user interactions and labels, and a network that enables fast inference. We showcase ScribblePrompt in an interactive demo, provide code, and release a dataset of scribble annotations at https://scribbleprompt.csail.mit.edu

Definition Search Book Streamline Icon: https://streamlinehq.com
References (151)
  1. Automatic segmentation of mandible in panoramic x-ray. Journal of Medical Imaging, 2(4):044003, 2015.
  2. Gradio: Hassle-free sharing and testing of ML models in the wild. CoRR, abs/1906.02569, 2019.
  3. Interactive Full Image Segmentation by Considering All Regions Jointly. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11614–11623, Long Beach, CA, USA, 2019. IEEE.
  4. Multimodal Neuroimaging in Schizophrenia: Description and Dissemination. Neuroinformatics, 15(4):343–364, 2017.
  5. Dataset of breast ultrasound images. Data in Brief, 28:104863, 2020.
  6. ECONet: Efficient Convolutional Online Likelihood Network for Scribble-based Interactive Segmentation. In International Conference on Medical Imaging with Deep Learning, pages 35–47. PMLR, 2022. arXiv:2201.04584 [cs, eess].
  7. Deep active learning for suggestive segmentation of biomedical image stacks via optimisation of Dice scores and traced boundary length. Medical Image Analysis, 81, 2022.
  8. Layer normalization. arXiv preprint arXiv:1607.06450, 2016.
  9. Error-Tolerant Scribbles Based Interactive Image Segmentation. In 2014 IEEE Conference on Computer Vision and Pattern Recognition, pages 392–399, Columbus, OH, USA, 2014. IEEE.
  10. The rsna-asnr-miccai brats 2021 benchmark on brain tumor segmentation and radiogenomic classification. arXiv preprint arXiv:2107.02314, 2021.
  11. Advancing the cancer genome atlas glioma mri collections with expert segmentation labels and radiomic features. Scientific data, 4(1):1–13, 2017.
  12. Deep placental vessel segmentation for fetoscopic mosaicking. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part III 23, pages 763–773. Springer, 2020.
  13. Large-scale interactive object segmentation with human annotators. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11700–11709, 2019.
  14. Deep learning techniques for automatic mri cardiac multi-structures segmentation and diagnosis: is the problem solved? IEEE transactions on medical imaging, 37(11):2514–2525, 2018.
  15. The liver tumor segmentation benchmark (lits). arXiv preprint arXiv:1901.04056, 2019.
  16. Nci-isbi 2013 challenge: automated segmentation of prostate structures. The Cancer Imaging Archive, 370(6):5, 2015.
  17. Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images. In Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001, pages 105–112, Vancouver, BC, Canada, 2001. IEEE Comput. Soc.
  18. Iterative interaction training for segmentation editing networks. In Machine Learning in Medical Imaging: 9th International Workshop, MLMI 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Proceedings 9, pages 363–370. Springer, 2018.
  19. Association of genomic subtypes of lower-grade gliomas with shape features automatically extracted by a deep learning algorithm. Computers in biology and medicine, 109:218–225, 2019.
  20. Universeg: Universal medical image segmentation. In ICCV, 2023.
  21. Nucleus segmentation across imaging experiments: the 2018 Data Science Bowl. Nature Methods, 16(12):1247–1253, 2019.
  22. An integrated micro-and macroarchitectural analysis of the drosophila brain by computer-assisted serial section electron microscopy. PLoS biology, 8(10):e1000502, 2010.
  23. Focalclick: Towards practical interactive image segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1300–1309, 2022.
  24. ScribbleSeg: Scribble-based Interactive Image Segmentation, 2023. arXiv:2303.11320 [cs].
  25. SAM-Med2D, 2023. arXiv:2308.16184 [cs].
  26. Skin lesion analysis toward melanoma detection: A challenge at the 2017 international symposium on biomedical imaging (isbi), hosted by the international skin imaging collaboration (ISIC). CoRR, abs/1710.05006, 2017.
  27. GeoS: Geodesic Image Segmentation. In Computer Vision – ECCV 2008, pages 99–112. Springer Berlin Heidelberg, Berlin, Heidelberg, 2008. Series Title: Lecture Notes in Computer Science.
  28. Neuralizer: General neuroimage analysis without re-training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6217–6230, 2023.
  29. Anatomical priors in convolutional networks for unsupervised biomedical segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 9290–9299, 2018.
  30. Teleophta: Machine learning and image processing methods for teleophthalmology. Irbm, 34(2):196–203, 2013.
  31. Early detection of myocardial infarction in low-quality echocardiography. IEEE Access, 9:34442–34453, 2021.
  32. Lee R Dice. Measures of the amount of ecologic association between species. Ecology, 26(3):297–302, 1945.
  33. Efficient graph-based image segmentation. International journal of computer vision, 59:167–181, 2004.
  34. Getting to 99% Accuracy in Interactive Segmentation, 2020. arXiv:2003.07932 [cs].
  35. Pannuke dataset extension, insights and baselines. arxiv. 2020 doi: 10.48550. ARXIV, 2003.
  36. Segmented anisotropic ssTEM dataset of neural tissue. 2013.
  37. The mcic collection: a shared repository of multi-modal, multi-site brain image data from a clinical investigation of schizophrenia. Neuroinformatics, 11:367–388, 2013.
  38. Automatic segmentation of brain mris of 2-year-olds into 83 regions of interest. Neuroimage, 40(2):672–684, 2008.
  39. Magnetic resonance imaging of the newborn brain: manual segmentation of labelled atlases in term-born and preterm infants. Neuroimage, 62(3):1499–1509, 2012.
  40. L. Grady. Random Walks for Image Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(11):1768–1783, 2006.
  41. Deep learning enables automatic detection and segmentation of brain metastases on multisequence mri. Journal of Magnetic Resonance Imaging, 51(1):175–182, 2020.
  42. Daniel Gut. X-ray images of the hip joints. 1, 2021. Publisher: Mendeley Data.
  43. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision, pages 1026–1034, 2015.
  44. Computer-vision benchmark segment-anything model (sam) in medical images: Accuracy in 12 datasets, 2023.
  45. The state of the art in kidney and kidney tumor segmentation in contrast-enhanced ct imaging: Results of the kits19 challenge. Medical Image Analysis, page 101821, 2020.
  46. Isles 2022: A multi-center magnetic resonance imaging stroke lesion segmentation dataset. Scientific data, 9(1):762, 2022.
  47. Learning the effect of registration hyperparameters with hypermorph. pages 1–30, 2022.
  48. Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response. IEEE Transactions on Medical imaging, 19(3):203–210, 2000.
  49. How to efficiently adapt large segmentation model(sam) to medical images, 2023.
  50. Segment anything model for medical images?, 2023.
  51. Humans in the Loop. Teeth segmentation dataset.
  52. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning, pages 448–456. pmlr, 2015.
  53. nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nature Methods, 18(2):203–211, 2021.
  54. Amos: A large-scale abdominal multi-organ benchmark for versatile medical image segmentation. arXiv preprint arXiv:2206.08023, 2022.
  55. Evaluation of current algorithms for segmentation of scar tissue from late gadolinium enhancement cardiovascular magnetic resonance of the left atrium: an open-access grand challenge. Journal of Cardiovascular Magnetic Resonance, 15(1):1–17, 2013.
  56. CHAOS - Combined (CT-MR) Healthy Abdominal Organ Segmentation Challenge Data, 2019.
  57. CHAOS Challenge - combined (CT-MR) healthy abdominal organ segmentation. Medical Image Analysis, 69:101950, 2021.
  58. Evaluation and improvement of segment anything model for interactive histopathology image segmentation, 2023.
  59. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  60. Left ventricular wall motion estimation by active polynomials for acute myocardial infarction detection. IEEE Access, 8:210301–210317, 2020.
  61. Segment anything. In ICCV, 2023.
  62. Tracked 3d ultrasound and deep neural network-based thyroid segmentation reduce interobserver variability in thyroid volumetry. Plos one, 17(7):e0268550, 2022.
  63. Standardized assessment of automatic segmentation of white matter hyperintensities and results of the wmh segmentation challenge. IEEE transactions on medical imaging, 38(11):2556–2568, 2019.
  64. A dynamic 4d probabilistic atlas of the developing brain. NeuroImage, 54(4):2750–2763, 2011.
  65. Segthor: segmentation of thoracic organs at risk in ct images. In 2020 Tenth International Conference on Image Processing Theory, Tools and Applications (IPTA), pages 1–6. IEEE, 2020.
  66. Miccai multi-atlas labeling beyond the cranial vault–workshop and challenge. In Proc. MICCAI Multi-Atlas Labeling Beyond Cranial Vault Workshop Challenge, page 12, 2015.
  67. Deep learning for segmentation using an open large-scale dataset in 2d echocardiography. IEEE transactions on medical imaging, 38(9):2198–2210, 2019.
  68. Computer-aided detection and diagnosis for prostate cancer based on mono and multi-parametric mri: a review. Computers in biology and medicine, 60:8–31, 2015.
  69. Ipn-v2 and octa-500: Methodology and dataset for retinal image segmentation. arXiv preprint arXiv:2012.07261, 2020.
  70. Interactive Image Segmentation with Latent Diversity. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 577–585, Salt Lake City, UT, 2018. IEEE.
  71. Scribblesup: Scribble-supervised convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3159–3167, 2016.
  72. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, pages 2980–2988, 2017.
  73. Samus: Adapting segment anything model for clinically-friendly and generalizable ultrasound image segmentation, 2023.
  74. Evaluation of prostate segmentation algorithms for mri: the promise12 challenge. Medical image analysis, 18(2):359–373, 2014.
  75. iSegFormer: Interactive Segmentation via Transformers with Application to 3D Knee MR Images. In Medical Image Computing and Computer Assisted Intervention (MICCAI), 2022.
  76. Simpleclick: Interactive image segmentation with simple vision transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 22290–22300, 2023.
  77. Annotated high-throughput microscopy image sets for validation. Nature methods, 9(7):637–637, 2012.
  78. A vertebral segmentation dataset with fracture grading. Radiology: Artificial Intelligence, 2(4):e190138, 2020.
  79. Word: Revisiting organs segmentation in the whole abdominal region. arXiv preprint arXiv:2111.02403, 2021a.
  80. MIDeepSeg: Minimally Interactive Segmentation of Unseen Objects from Medical Images Using Deep Learning. Medical Image Analysis, 72:102102, 2021b.
  81. Fast and low-gpu-memory abdomen ct organ segmentation: the flare challenge. Medical Image Analysis, 82:102616, 2022.
  82. Segment anything in medical images, 2023.
  83. Rose: a retinal oct-angiography vessel segmentation dataset and new model. IEEE Transactions on Medical Imaging, 40(3):928–939, 2021.
  84. Duke liver dataset (MRI) v2, 2023.
  85. Deep Extreme Cut: From Extreme Points to Object Segmentation, 2018. arXiv:1711.09081.
  86. Open access series of imaging studies (oasis): cross-sectional mri data in young, middle aged, nondemented, and demented older adults. Journal of cognitive neuroscience, 19(9):1498–1507, 2007.
  87. The parkinson progression marker initiative (ppmi). Progress in neurobiology, 95(4):629–635, 2011.
  88. Deep learning segmentation of transverse musculoskeletal ultrasound images for neuromuscular disease assessment. Computers in Biology and Medicine, 135:104623, 2021.
  89. Radiogenomics of lower-grade glioma: algorithmically-assessed tumor shape is associated with tumor genomic subtypes and patient outcomes in a multi-institutional study with the cancer genome atlas data. Journal of neuro-oncology, 133:27–35, 2017.
  90. Segment anything model for medical image analysis: An experimental study. Medical Image Analysis, 89:102918, 2023.
  91. Optical coherence tomography for brain imaging and developmental biology. IEEE Journal of Selected Topics in Quantum Electronics, 22(4):120–132, 2015.
  92. Quantification of uncertainties in biomedical image quantification 2021. 4th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2021), 2021.
  93. The multimodal brain tumor image segmentation benchmark (brats). IEEE transactions on medical imaging, 34(10):1993–2024, 2014.
  94. V-net: Fully convolutional neural networks for volumetric medical image segmentation. Fourth International Conference on 3D Vision (3DV), pages 565–571, 2016.
  95. Ultrasound nerve segmentation, 2016.
  96. Interactive Blood Vessel Segmentation from Retinal Fundus Image Based on Canny Edge Detector. Sensors, 21(19):6380, 2021.
  97. Adaptivesam: Towards efficient tuning of sam for surgical scene segmentation. arXiv preprint arXiv:2308.03726, 2023.
  98. An automatic multi-tissue human fetal brain segmentation benchmark using the fetal tissue annotation dataset. Scientific Data, 8(1):1–14, 2021.
  99. An open access thyroid ultrasound image database. In 10th international symposium on medical information processing and analysis, page 92870W. SPIE / International Society for Optics and Photonics, 2015.
  100. Ril-contour: a medical imaging dataset annotation tool for and with deep learning. Journal of digital imaging, 32:571–581, 2019.
  101. HaN-Seg: The head and neck organ-at-risk CT and MR segmentation dataset. Medical Physics, 50(3):1917–1927, 2023. tex.eprint: https://aapm.onlinelibrary.wiley.com/doi/pdf/10.1002/mp.16197.
  102. Indian diabetic retinopathy image dataset (idrid), 2018.
  103. Evaluation framework for algorithms segmenting short axis cardiac mri. The MIDAS Journal-Cardiac MR Left Ventricle Segmentation Challenge, 49, 2009.
  104. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748–8763. PMLR, 2021.
  105. Transfusion: Understanding transfer learning for medical imaging. Advances in neural information processing systems, 32, 2019.
  106. DeepCut: Object Segmentation From Bounding Box Annotations Using Convolutional Neural Networks. IEEE Transactions on Medical Imaging, 36(2):674–683, 2017.
  107. CT-ORG, a new dataset for multiple organ segmentation in computed tomography. Scientific Data, 7(1):381, 2020.
  108. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pages 234–241. Springer, 2015.
  109. Going to Extremes: Weakly Supervised Medical Image Segmentation. Machine Learning and Knowledge Extraction, 3(2):507–524, 2021.
  110. “GrabCut” — Interactive Foreground Extraction using Iterated Graph Cuts. ACM Transactions on Graphics, 23:309–314, 2004.
  111. Interactive segmentation of medical images through fully convolutional neural networks, 2019. arXiv:1903.08205.
  112. Deep learning saliency maps do not accurately highlight diagnostically relevant regions for medical image interpretation. MedRxiv, 2021.
  113. Flair*: a combined mr contrast technique for visualizing white matter lesions and parenchymal veins. Radiology, 265(3):926–932, 2012.
  114. Detailed annotations of chest x-rays via ct projection for report understanding. In Proceedings of the 33th British Machine Vision Conference (BMVC), 2022.
  115. Construction of a consistent high-definition spatio-temporal atlas of the developing brain using adaptive kernel regression. Neuroimage, 59(3):2255–2265, 2012.
  116. Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the luna16 challenge. Medical image analysis, 42:1–13, 2017.
  117. Generalist vision foundation models for medical imaging: A case study of segment anything model on zero-shot medical segmentation. Diagnostics, 13(11):1947, 2023.
  118. A large annotated medical image dataset for the development and evaluation of segmentation algorithms. arXiv preprint arXiv:1902.09063, 2019.
  119. F-BRS: Rethinking Backpropagating Refinement for Interactive Segmentation. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8620–8629, Seattle, WA, USA, 2020. IEEE.
  120. Reviving Iterative Training with Mask Guidance for Interactive Segmentation, 2021. arXiv:2102.06583 [cs].
  121. CT2US: Cross-modal transfer learning for kidney segmentation in ultrasound images with synthesized data. Ultrasonics, 122:106706, 2022.
  122. Ridge-based vessel segmentation in color images of the retina. IEEE transactions on medical imaging, 23(4):501–509, 2004.
  123. Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysis. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 20698–20708, New Orleans, LA, USA, 2022. IEEE.
  124. Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv:1607.08022, 2016.
  125. Segmentation of anatomical structures in chest radiographs using supervised methods: a comparative study on a public database. Medical Image Analysis, 10(1):19–40, 2006.
  126. “GrowCut” - Interactive Multi-Label N-D Image Segmentation By Cellular Automata. 1(4):150–156, 2005.
  127. Improving realism in patient-specific abdominal ultrasound simulation using cyclegans. International journal of computer assisted radiology and surgery, 15(2):183–192, 2020.
  128. Sam-octa: A fine-tuning strategy for applying foundation model to octa image segmentation tasks, 2023a.
  129. Slic-Seg: A minimally interactive segmentation of the placenta from sparse and motion-corrupted fetal MRI in multiple views. Medical Image Analysis, 34:137–147, 2016.
  130. Interactive Medical Image Segmentation Using Deep Learning With Image-Specific Fine Tuning. IEEE Transactions on Medical Imaging, 37(7):1562–1573, 2018.
  131. DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(7):1559–1572, 2019.
  132. Images speak in images: A generalist painter for in-context visual learning. arXiv preprint arXiv:2212.02499, 2022.
  133. SegGPT: Segmenting Everything In Context, 2023b. arXiv:2304.03284 [cs].
  134. Totalsegmentator: Robust segmentation of 104 anatomic structures in ct images. Radiology: Artificial Intelligence, 5(5), 2023.
  135. Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation, 2023. arXiv:2304.12620 [cs].
  136. Group normalization. In Proceedings of the European conference on computer vision (ECCV), pages 3–19, 2018.
  137. Deep Interactive Object Selection. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 373–381, Las Vegas, NV, USA, 2016. IEEE.
  138. Deep GrabCut for Object Selection. arXiv, 2017a. arXiv:1707.00243 [cs].
  139. Deep GrabCut for Object Selection. arXiv, 2017b. arXiv:1707.00243.
  140. Sa-med2d-20m dataset: Segment anything in 2d medical imaging with 20 million masks. arXiv preprint arXiv:2311.11969, 2023.
  141. User-guided 3d active contour segmentation of anatomical structures: significantly improved efficiency and reliability. Neuroimage, 31(3):1116–1128, 2006.
  142. Semantic segmentation of covid-19 lesions with a multiscale dilated convolutional network. Scientific Reports, 12(1):1847, 2022a.
  143. Customized segment anything model for medical image segmentation, 2023.
  144. Interactive Object Segmentation With Inside-Outside Guidance. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12231–12241. IEEE, 2020.
  145. A fast parallel algorithm for thinning digital patterns. Commun. ACM, 27(3):236–239, 1984.
  146. Busis: A benchmark for breast ultrasound image segmentation. In Healthcare, page 729. MDPI, 2022b.
  147. A multi-modality ovarian tumor ultrasound image dataset for unsupervised cross-domain semantic segmentation. CoRR, abs/2207.06799, 2022.
  148. Evaluation and comparison of 3d intervertebral disc localization and segmentation methods for 3d t2 mr data: A grand challenge. Medical image analysis, 35:327–344, 2017.
  149. Fast and robust segmentation of white blood cell images by self-supervised learning. Micron, 107:55–71, 2018.
  150. Volumetric memory network for interactive medical image segmentation. Medical Image Analysis, 83:102599, 2023.
  151. Segment Everything Everywhere All at Once, 2023. arXiv:2304.06718 [cs].
Citations (5)
View on Semantic Scholar

Summary

  • The paper introduces a novel interactive segmentation framework that leverages scribble-based inputs for enhanced biomedical image analysis.
  • It employs a diverse training strategy using 65 biomedical datasets with both real and synthetic labels to ensure robust performance.
  • Evaluations show significant improvements with a 28% reduction in annotation time and a 15% increase in segmentation accuracy.

Interactive Segmentation Advances in Medical Imaging

Medical imaging plays a pivotal role in clinical care and scientific research by allowing detailed visualization of internal body structures. While automated medical image segmentation can offer robust assistance, the process still relies on manual annotation for many tasks, which is time-consuming and requires expertise. Addressing this bottleneck, a new framework, ScribblePrompt, emerges as a game-changer in the field of medical imaging by offering a fast and flexible interactive segmentation system adaptable to a wide range of medical images.

Introducing ScribblePrompt

ScribblePrompt innovates in the field of interactive segmentation by supporting scribble-based inputs, in addition to traditional clicks and bounding boxes. These scribbles are an intuitive form of interaction, enabling more detailed guidance for complex objects with minimal effort increase over clicks. To effectively train models to respond to these types of inputs, ScribblePrompt incorporates algorithms for simulating realistic user interactions, thereby allowing the model to interpret scribbles, clicks, and bounding boxes with high efficacy.

Training on Diverse Data

A key feature of ScribblePrompt's robustness is its training on a diverse collection of biomedical datasets. The framework uses both real and synthetic labels to train across 65 different biomedical imaging datasets, which promotes an unprecedented level of generalization across various medical segmentation tasks. The use of synthetic labels is especially noteworthy as it aids in preventing the model from overfitting to specific tasks, which is a common challenge in machine learning.

Performance and Efficiency

Evaluations illustrate that ScribblePrompt surpasses established methods in accuracy for a majority of tasks and exhibits real-time performance enabled even on a single CPU. In user tests, ScribblePrompt demonstrated a reduction in annotation time by 28% while improving segmentation accuracy, as measured by the Dice coefficient, by 15% compared to traditional methods.

Release to the Community

In the spirit of open science and collaboration, the team behind ScribblePrompt has released an interactive tool, relevant code, and model weights for community use. This initiative could significantly streamline workflows in medical imaging and potentially catalyze further developments in the automation of medical image analysis.

User Experience Validation

In a user paper involving experienced annotators, ScribblePrompt was preferred due to its ability to segment accurately based on diverse input types, its responsiveness to corrections, and the reduced effort to achieve quality segmentations. This observational paper validates the practical usefulness and potential for ScribblePrompt to relieve the burden of manual segmentation within medical imaging tasks.

In conclusion, with the advent of ScribblePrompt, there's an anticipation of a transformative shift in medical segmentation approaches. Its capability to combine intuitive interaction methods with high adaptability and performance offers a promising tool for both clinical and research applications, marking a significant step forward in interactive medical image segmentation.

Youtube Logo Streamline Icon: https://streamlinehq.com

YouTube