Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Segment Any Medical Model Extended (2403.18114v1)

Published 26 Mar 2024 in cs.CV

Abstract: The Segment Anything Model (SAM) has drawn significant attention from researchers who work on medical image segmentation because of its generalizability. However, researchers have found that SAM may have limited performance on medical images compared to state-of-the-art non-foundation models. Regardless, the community sees potential in extending, fine-tuning, modifying, and evaluating SAM for analysis of medical imaging. An increasing number of works have been published focusing on the mentioned four directions, where variants of SAM are proposed. To this end, a unified platform helps push the boundary of the foundation model for medical images, facilitating the use, modification, and validation of SAM and its variants in medical image segmentation. In this work, we introduce SAMM Extended (SAMME), a platform that integrates new SAM variant models, adopts faster communication protocols, accommodates new interactive modes, and allows for fine-tuning of subcomponents of the models. These features can expand the potential of foundation models like SAM, and the results can be translated to applications such as image-guided therapy, mixed reality interaction, robotic navigation, and data augmentation.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (59)
  1. Ranjbarzadeh, R., Bagherian Kasgari, A., Jafarzadeh Ghoushchi, S., Anari, S., Naseri, M., and Bendechache, M., “Brain tumor segmentation based on deep learning and an attention mechanism using mri multi-modalities brain images,” Scientific Reports 11(1), 10930 (2021).
  2. Gan, Y., Xia, Z., Xiong, J., Li, G., and Zhao, Q., “Tooth and alveolar bone segmentation from dental computed tomography images,” IEEE journal of biomedical and health informatics 22(1), 196–204 (2017).
  3. Murphy, R. J., Basafa, E., Hashemi, S., Grant, G. T., Liacouras, P., Susarla, S. M., Otake, Y., Santiago, G., Armand, M., and Gordon, C. R., “Optimizing hybrid occlusion in face-jaw-teeth transplantation: a preliminary assessment of real-time cephalometry as part of the computer-assisted planning and execution workstation for craniomaxillofacial surgery,” Plastic and reconstructive surgery 136(2), 350 (2015).
  4. Khan, A. R., Wang, L., and Beg, M. F., “Freesurfer-initiated fully-automated subcortical brain segmentation in mri using large deformation diffeomorphic metric mapping,” Neuroimage 41(3), 735–746 (2008).
  5. Hashemi, S. R., Salehi, S. S. M., Erdogmus, D., Prabhu, S. P., Warfield, S. K., and Gholipour, A., “Asymmetric loss functions and deep densely-connected networks for highly-imbalanced medical image segmentation: Application to multiple sclerosis lesion detection,” IEEE Access 7, 1721–1735 (2018).
  6. Shrivastava, D., Sanyal, S., Maji, A. K., and Kandar, D., “Bone cancer detection using machine learning techniques,” in [Smart Healthcare for Disease Diagnosis and Prevention ], 175–183, Elsevier (2020).
  7. Radaelli, A. and Peiro, J., “On the segmentation of vascular geometries from medical images,” International Journal for Numerical Methods in Biomedical Engineering 26(1), 3–34 (2010).
  8. Grupp, R. B., Unberath, M., Gao, C., Hegeman, R. A., Murphy, R. J., Alexander, C. P., Otake, Y., McArthur, B. A., Armand, M., and Taylor, R. H., “Automatic annotation of hip anatomy in fluoroscopy for robust and efficient 2d/3d registration,” International journal of computer assisted radiology and surgery 15, 759–769 (2020).
  9. Otake, Y., Armand, M., Armiger, R. S., Kutzer, M. D., Basafa, E., Kazanzides, P., and Taylor, R. H., “Intraoperative image-based multiview 2d/3d registration for image-guided orthopaedic surgery: incorporation of fiducial-based c-arm tracking and gpu-acceleration,” IEEE transactions on medical imaging 31(4), 948–962 (2011).
  10. Unberath, M., Zaech, J.-N., Lee, S. C., Bier, B., Fotouhi, J., Armand, M., and Navab, N., “Deepdrr–a catalyst for machine learning in fluoroscopy-guided procedures,” in [Medical Image Computing and Computer Assisted Intervention–MICCAI 2018: 21st International Conference, Granada, Spain, September 16-20, 2018, Proceedings, Part IV 11 ], 98–106, Springer (2018).
  11. Gao, C., Phalen, H., Sefati, S., Ma, J., Taylor, R. H., Unberath, M., and Armand, M., “Fluoroscopic navigation for a surgical robotic system including a continuum manipulator,” IEEE Transactions on Biomedical Engineering 69(1), 453–464 (2021).
  12. Canny, J., “A computational approach to edge detection,” IEEE Transactions on pattern analysis and machine intelligence (6), 679–698 (1986).
  13. Tang, H., Wu, E., Ma, Q., Gallagher, D., Perera, G., and Zhuang, T., “Mri brain image segmentation by multi-resolution edge detection and region selection,” Computerized Medical Imaging and Graphics 24(6), 349–357 (2000).
  14. Ng, H., Ong, S., Foong, K., Goh, P.-S., and Nowinski, W., “Medical image segmentation using k-means clustering and improved watershed algorithm,” in [2006 IEEE southwest symposium on image analysis and interpretation ], 61–65, IEEE (2006).
  15. Chudasama, D., Patel, T., Joshi, S., and Prajapati, G. I., “Image segmentation using morphological operations,” International Journal of Computer Applications 117(18) (2015).
  16. Khadidos, A., Sanchez, V., and Li, C.-T., “Weighted level set evolution based on local edge features for medical image segmentation,” IEEE Transactions on Image Processing 26(4), 1979–1991 (2017).
  17. Pratondo, A., Chui, C.-K., and Ong, S.-H., “Integrating machine learning with region-based active contour models in medical image segmentation,” Journal of Visual Communication and Image Representation 43, 1–9 (2017).
  18. Li, X., Chen, H., Qi, X., Dou, Q., Fu, C.-W., and Heng, P.-A., “H-denseunet: hybrid densely connected unet for liver and tumor segmentation from ct volumes,” IEEE transactions on medical imaging 37(12), 2663–2674 (2018).
  19. Cao, H., Wang, Y., Chen, J., Jiang, D., Zhang, X., Tian, Q., and Wang, M., “Swin-unet: Unet-like pure transformer for medical image segmentation,” in [European conference on computer vision ], 205–218, Springer (2022).
  20. Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., Lu, L., Yuille, A. L., and Zhou, Y., “Transunet: Transformers make strong encoders for medical image segmentation,” arXiv preprint arXiv:2102.04306 (2021).
  21. Wang, R., Lei, T., Cui, R., Zhang, B., Meng, H., and Nandi, A. K., “Medical image segmentation using deep learning: A survey,” IET Image Processing 16(5), 1243–1267 (2022).
  22. Wang, E. K., Chen, C.-M., Hassan, M. M., and Almogren, A., “A deep learning based medical image segmentation technique in internet-of-medical-things domain,” Future Generation Computer Systems 108, 135–144 (2020).
  23. Gao, C., Killeen, B. D., Hu, Y., Grupp, R. B., Taylor, R. H., Armand, M., and Unberath, M., “Synthetic data accelerates the development of generalizable learning-based algorithms for X-ray image analysis,” Nat Mach Intell 5, 294–308 (Mar. 2023).
  24. Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.-Y., et al., “Segment anything,” arXiv preprint arXiv:2304.02643 (2023).
  25. Xian, Y., Lampert, C. H., Schiele, B., and Akata, Z., “Zero-shot learning—a comprehensive evaluation of the good, the bad and the ugly,” IEEE transactions on pattern analysis and machine intelligence 41(9), 2251–2265 (2018).
  26. Liu, Y., Zhang, J., She, Z., Kheradmand, A., and Armand, M., “Samm (segment any medical model): A 3d slicer integration to sam,” arXiv preprint arXiv:2304.05622 (2023).
  27. Pieper, S., Halle, M., and Kikinis, R., “3d slicer,” in [2004 2nd IEEE international symposium on biomedical imaging: nano to macro (IEEE Cat No. 04EX821) ], 632–635, IEEE (2004).
  28. Zhang, Y. and Jiao, R., “How segment anything model (sam) boost medical image segmentation?,” arXiv preprint arXiv:2305.03678 (2023).
  29. Mattjie, C., de Moura, L. V., Ravazio, R. C., Kupssinskü, L. S., Parraga, O., Delucis, M. M., and Barros, R. C., “Exploring the zero-shot capabilities of the segment anything model (sam) in 2d medical imaging: A comprehensive evaluation and practical guideline,” arXiv preprint arXiv:2305.00109 (2023).
  30. Hu, X., Xu, X., and Shi, Y., “How to efficiently adapt large segmentation model (sam) to medical images,” arXiv preprint arXiv:2306.13731 (2023).
  31. Shi, X., Chai, S., Li, Y., Cheng, J., Bai, J., Zhao, G., and Chen, Y.-W., “Cross-modality attention adapter: A glioma segmentation fine-tuning method for sam using multimodal brain mr images,” arXiv preprint arXiv:2307.01124 (2023).
  32. Kellener, E., Nath, I., Ngo, A., Nguyen, T., Schuman, J., Adler, C., and Kartikeya, A., “Utilizing segment anything model for assessing localization of grad-cam in medical imaging,” arXiv preprint arXiv:2306.15692 (2023).
  33. Deng, R., Cui, C., Liu, Q., Yao, T., Remedios, L. W., Bao, S., Landman, B. A., Wheless, L. E., Coburn, L. A., Wilson, K. T., et al., “Segment anything model (sam) for digital pathology: Assess zero-shot segmentation on whole slide imaging,” arXiv preprint arXiv:2304.04155 (2023).
  34. Roy, S., Wald, T., Koehler, G., Rokuss, M. R., Disch, N., Holzschuh, J., Zimmerer, D., and Maier-Hein, K. H., “Sam. md: Zero-shot medical image segmentation capabilities of the segment anything model,” arXiv preprint arXiv:2304.05396 (2023).
  35. Mazurowski, M. A., Dong, H., Gu, H., Yang, J., Konz, N., and Zhang, Y., “Segment anything model for medical image analysis: an experimental study,” Medical Image Analysis 89, 102918 (2023).
  36. Shi, P., Qiu, J., Abaxi, S. M. D., Wei, H., Lo, F. P.-W., and Yuan, W., “Generalist vision foundation models for medical imaging: A case study of segment anything model on zero-shot medical segmentation,” Diagnostics 13(11), 1947 (2023).
  37. Huang, Y., Yang, X., Liu, L., Zhou, H., Chang, A., Zhou, X., Chen, R., Yu, J., Chen, J., Chen, C., et al., “Segment anything model for medical images?,” Medical Image Analysis , 103061 (2023).
  38. Cheng, D., Qin, Z., Jiang, Z., Zhang, S., Lao, Q., and Li, K., “Sam on medical images: A comprehensive study on three prompt modes,” arXiv preprint arXiv:2305.00035 (2023).
  39. Zhang, C., Han, D., Qiao, Y., Kim, J. U., Bae, S.-H., Lee, S., and Hong, C. S., “Faster segment anything: Towards lightweight sam for mobile applications,” arXiv preprint arXiv:2306.14289 (2023).
  40. Bai, X. and Xia, Y., “Sam++: Enhancing anatomic matching using semantic information and structural inference,” arXiv preprint arXiv:2306.13988 (2023).
  41. Chen, T., Zhu, L., Ding, C., Cao, R., Zhang, S., Wang, Y., Li, Z., Sun, L., Mao, P., and Zang, Y., “Sam fails to segment anything?–sam-adapter: Adapting sam in underperformed scenes: Camouflage, shadow, and more,” arXiv preprint arXiv:2304.09148 (2023).
  42. Qiu, Z., Hu, Y., Li, H., and Liu, J., “Learnable ophthalmology sam,” arXiv preprint arXiv:2304.13425 (2023).
  43. Shaharabany, T., Dahan, A., Giryes, R., and Wolf, L., “Autosam: Adapting sam to medical images by overloading the prompt encoder,” arXiv preprint arXiv:2306.06370 (2023).
  44. Ma, J. and Wang, B., “Segment anything in medical images,” arXiv preprint arXiv:2304.12306 (2023).
  45. Paranjape, J. N., Nair, N. G., Sikder, S., Vedula, S. S., and Patel, V. M., “Adaptivesam: Towards efficient tuning of sam for surgical scene segmentation,” arXiv preprint arXiv:2308.03726 (2023).
  46. Huang, Z., Liu, H., Zhang, H., Xing, F., Laine, A., Angelini, E., Hendon, C., and Gan, Y., “Push the boundary of sam: A pseudo-label correction framework for medical segmentation,” arXiv preprint arXiv:2308.00883 (2023).
  47. Zhang, K. and Liu, D., “Customized segment anything model for medical image segmentation,” arXiv preprint arXiv:2304.13785 (2023).
  48. Deng, G., Zou, K., Ren, K., Wang, M., Yuan, X., Ying, S., and Fu, H., “Sam-u: Multi-box prompts triggered uncertainty estimation for reliable sam in medical image,” arXiv preprint arXiv:2307.04973 (2023).
  49. Zhang, Y., Zhou, T., Wang, S., Liang, P., Zhang, Y., and Chen, D. Z., “Input augmentation with sam: Boosting medical image segmentation with segmentation foundation model,” in [International Conference on Medical Image Computing and Computer-Assisted Intervention ], 129–139, Springer (2023).
  50. Lei, W., Wei, X., Zhang, X., Li, K., and Zhang, S., “Medlsam: Localize and segment anything model for 3d medical images,” arXiv preprint arXiv:2306.14752 (2023).
  51. Wang, C., Li, D., Wang, S., Zhang, C., Wang, Y., Liu, Y., and Yang, G., “Sammed: A medical image annotation framework based on large vision model,” arXiv preprint arXiv:2307.05617 (2023).
  52. Diaz-Pinto, A., Alle, S., Nath, V., Tang, Y., Ihsani, A., Asad, M., Pérez-García, F., Mehta, P., Li, W., Flores, M., et al., “Monai label: A framework for ai-assisted interactive labeling of 3d medical images,” arXiv preprint arXiv:2203.12362 (2022).
  53. Liu, Y., Kheradmand, A., and Armand, M., “Toward process controlled medical robotic system,” arXiv preprint arXiv:2308.05809 (2023).
  54. Li, H., Yang, Y., Ji, Y., Peng, W., Martin-Gomez, A., Yan, W., Qian, L., Ding, H., Zhao, Z., and Wang, G., “3d slicer-ar-bridge: 3d slicer ar connection for medical image visualization and interaction with ar-hmd,” in [2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) ], 399–404, IEEE (2023).
  55. Li, H., Yan, W., Liu, D., Qian, L., Yang, Y., Liu, Y., Zhao, Z., Ding, H., and Wang, G., “Evd surgical guidance with retro-reflective tool tracking and spatial reconstruction using head-mounted augmented reality device,” arXiv preprint arXiv:2306.15490 (2023).
  56. Fedorov, A., Beichel, R., Kalpathy-Cramer, J., Finet, J., Fillion-Robin, J.-C., Pujol, S., Bauer, C., Jennings, D., Fennessy, F., Sonka, M., et al., “3d slicer as an image computing platform for the quantitative imaging network,” Magnetic resonance imaging 30(9), 1323–1341 (2012).
  57. Pinter, C., Lasso, A., Wang, A., Jaffray, D., and Fichtinger, G., “Slicerrt: radiation therapy research toolkit for 3d slicer,” Medical physics 39(10), 6332–6338 (2012).
  58. Kikinis, R., Pieper, S. D., and Vosburgh, K. G., “3d slicer: a platform for subject-specific image analysis, visualization, and clinical support,” in [Intraoperative imaging and image-guided therapy ], 277–289, Springer (2013).
  59. Schroeder, W. J., Avila, L. S., and Hoffman, W., “Visualizing with vtk: a tutorial,” IEEE Computer graphics and applications 20(5), 20–27 (2000).
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Yihao Liu (85 papers)
  2. Jiaming Zhang (117 papers)
  3. Andres Diaz-Pinto (8 papers)
  4. Haowei Li (25 papers)
  5. Alejandro Martin-Gomez (9 papers)
  6. Amir Kheradmand (7 papers)
  7. Mehran Armand (51 papers)
Citations (7)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now