Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

SAMCT: Segment Any CT Allowing Labor-Free Task-Indicator Prompts (2403.13258v1)

Published 20 Mar 2024 in cs.CV

Abstract: Segment anything model (SAM), a foundation model with superior versatility and generalization across diverse segmentation tasks, has attracted widespread attention in medical imaging. However, it has been proved that SAM would encounter severe performance degradation due to the lack of medical knowledge in training and local feature encoding. Though several SAM-based models have been proposed for tuning SAM in medical imaging, they still suffer from insufficient feature extraction and highly rely on high-quality prompts. In this paper, we construct a large CT dataset consisting of 1.1M CT images and 5M masks from public datasets and propose a powerful foundation model SAMCT allowing labor-free prompts. Specifically, based on SAM, SAMCT is further equipped with a U-shaped CNN image encoder, a cross-branch interaction module, and a task-indicator prompt encoder. The U-shaped CNN image encoder works in parallel with the ViT image encoder in SAM to supplement local features. Cross-branch interaction enhances the feature expression capability of the CNN image encoder and the ViT image encoder by exchanging global perception and local features from one to the other. The task-indicator prompt encoder is a plug-and-play component to effortlessly encode task-related indicators into prompt embeddings. In this way, SAMCT can work in an automatic manner in addition to the semi-automatic interactive strategy in SAM. Extensive experiments demonstrate the superiority of SAMCT against the state-of-the-art task-specific and SAM-based medical foundation models on various tasks. The code, data, and models are released at https://github.com/xianlin7/SAMCT.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (56)
  1. The medical segmentation decathlon. Nature Communications, 13(1): 4128.
  2. The liver tumor segmentation benchmark (LiTS). Medical Image Analysis, 84: 102680.
  3. TongueSAM: An universal tongue segmentation model based on SAM with zero-shot. arXiv preprint arXiv:2308.06444.
  4. AAU-net: an adaptive attention U-net for breast lesions segmentation in ultrasound images. IEEE Transactions on Medical Imaging.
  5. Transunet: Transformers make strong encoders for medical image segmentation. arXiv preprint arXiv:2102.04306.
  6. SAM-Med2D. arXiv preprint arXiv:2308.16184.
  7. CPFNet: Context pyramid fusion network for medical image segmentation. IEEE Transactions on Medical Imaging, 39(10): 3008–3018.
  8. CA-Net: Comprehensive attention convolutional neural networks for explainable medical image segmentation. IEEE Transactions on Medical Imaging, 40(2): 699–711.
  9. Swin unetr: Swin transformers for semantic segmentation of brain tumors in mri images. In International MICCAI Brainlesion Workshop, 272–284. Springer.
  10. Unetr: Transformers for 3d medical image segmentation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 574–584.
  11. H2Former: An efficient hierarchical hybrid transformer for medical image segmentation. IEEE Transactions on Medical Imaging.
  12. Accuracy of segment-anything model (sam) in medical image segmentation tasks. arXiv preprint arXiv:2304.09324.
  13. The KiTS21 challenge: Automatic segmentation of kidneys, renal tumors, and renal cysts in corticomedullary-phase CT. arXiv preprint arXiv:2307.01984.
  14. Computed tomography images for intracranial hemorrhage detection and segmentation. Intracranial Hemorrhage Segmentation Using A Deep Convolutional Model. Data, 5(1): 14.
  15. Missformer: An effective transformer for 2d medical image segmentation. IEEE Transactions on Medical Imaging.
  16. Segment anything model for medical images? arXiv preprint arXiv:2304.14660.
  17. Igor.Slinko. 2020. COVID-19 CT images segmentation.
  18. nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nature Methods, 18(2): 203–211.
  19. AMOS: A large-scale abdominal multi-organ benchmark for versatile medical image segmentation. arXiv preprint arXiv:2206.08023.
  20. Pediatric chest/abdomen/pelvic CT exams with expert organ contours (Pediatric-CT-SEG).
  21. An introduction to PET-CT imaging. Radiographics, 24(2): 523–543.
  22. CHAOS challenge-combined (CT-MR) healthy abdominal organ segmentation. Medical Image Analysis, 69: 101950.
  23. Segment anything. arXiv preprint arXiv:2304.02643.
  24. MICCAI multi-atlas labeling beyond the cranial vault–workshop and challenge. In Proc. MICCAI Multi-Atlas Labeling Beyond Cranial Vault—Workshop Challenge, volume 5, 12.
  25. Auto-prompting SAM for mobile friendly 3D medical image segmentation. arXiv preprint arXiv:2308.14936.
  26. Automatic structure segmentation for radio therapy planning challenge 2020. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Lima, Peru, 4–8.
  27. The state-of-the-art 3D anisotropic intracranial hemorrhage segmentation on non-contrast head CT: The INSTANCE challenge. arXiv preprint arXiv:2301.03281.
  28. Polyp-sam: Transfer sam for polyp segmentation. arXiv preprint arXiv:2305.00293.
  29. WORD: A large scale dataset, benchmark and clinical applicable study for abdominal organ segmentation from CT image. Medical Image Analysis, 82: 102642–102642.
  30. Segment anything in medical images. arXiv preprint arXiv:2304.12306.
  31. Toward data-efficient learning: A benchmark for COVID-19 CT lung and infection segmentation. Medical Physics, 48(3): 1197–1210.
  32. A new dataset of computed-tomography angiography images for computer-aided detection of pulmonary embolism. Scientific Data, 5(1): 1–9.
  33. Segment anything model for medical image analysis: an experimental study. Medical Image Analysis, 89: 102918.
  34. Image segmentation using deep learning: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(7): 3523–3542.
  35. Voxel-level segmentation of pathologically-proven Adrenocortical carcinoma with Ki-67 expression (Adrenal-ACC-Ki67-Seg).
  36. Foundation models for generalist medical artificial intelligence. Nature, 616(7956): 259–265.
  37. MosMedData: Chest CT scans with COVID-19 related findings dataset. arXiv preprint arXiv:2005.06465.
  38. LNDb: A lung nodule database on computed tomography. arXiv preprint arXiv:1911.08434.
  39. HaN-Seg: The head and neck organ-at-risk CT and MR segmentation dataset. Medical Physics, 50(3): 1917–1927.
  40. Evaluation of segmentation methods on head and neck CT: Auto-segmentation challenge 2015. Medical Physics, 44(5): 2020–2036.
  41. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention, 234–241. Springer.
  42. Data from pancreas-CT.
  43. Rapid artificial intelligence solutions in a pandemic—The COVID-19-20 lung CT lesion segmentation challenge. Medical Image Analysis, 82: 102605.
  44. Mednext: transformer-driven scaling of convnets for medical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 405–415. Springer.
  45. AutoSAM: Adapting SAM to medical images by overloading the prompt encoder. arXiv preprint arXiv:2306.06370.
  46. Stress-testing pelvic autosegmentation algorithms using anatomical edge cases (Prostate Anatomical Edge Cases).
  47. TotalSegmentator: Robust segmentation of 104 anatomical structures in CT images. arXiv preprint arXiv:2208.05868.
  48. Medical sam adapter: Adapting segment anything model for medical image segmentation. arXiv preprint arXiv:2304.12620.
  49. Data from lung CT segmentation challenge (LCTSC).
  50. SurgicalSAM: Efficient class promptable surgical instrument segmentation. arXiv preprint arXiv:2308.08746.
  51. Customized segment anything model for medical image segmentation. arXiv preprint arXiv:2304.13785.
  52. Multi-site, multi-domain airway tree modeling. Medical Image Analysis, 90: 102957.
  53. UV-SAM: Adapting segment anything model for urban village identification. arXiv preprint arXiv:2401.08083.
  54. Transfuse: Fusing transformers and cnns for medical image segmentation. In Medical Image Computing and Computer Assisted Intervention, 14–24. Springer.
  55. nnFormer: Volumetric medical image segmentation via a 3D transformer. IEEE Transactions on Image Processing.
  56. Evaluation of algorithms for multi-modality whole heart segmentation: An open-access grand challenge. Medical Image Analysis, 58: 101537.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Xian Lin (13 papers)
  2. Yangyang Xiang (3 papers)
  3. Zhehao Wang (2 papers)
  4. Kwang-Ting Cheng (96 papers)
  5. Zengqiang Yan (20 papers)
  6. Li Yu (193 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Github Logo Streamline Icon: https://streamlinehq.com

GitHub

  1. GitHub - xianlin7/SAMCT (23 stars)
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets