Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

I-MedSAM: Implicit Medical Image Segmentation with Segment Anything (2311.17081v3)

Published 28 Nov 2023 in cs.CV

Abstract: With the development of Deep Neural Networks (DNNs), many efforts have been made to handle medical image segmentation. Traditional methods such as nnUNet train specific segmentation models on the individual datasets. Plenty of recent methods have been proposed to adapt the foundational Segment Anything Model (SAM) to medical image segmentation. However, they still focus on discrete representations to generate pixel-wise predictions, which are spatially inflexible and scale poorly to higher resolution. In contrast, implicit methods learn continuous representations for segmentation, which is crucial for medical image segmentation. In this paper, we propose I-MedSAM, which leverages the benefits of both continuous representations and SAM, to obtain better cross-domain ability and accurate boundary delineation. Since medical image segmentation needs to predict detailed segmentation boundaries, we designed a novel adapter to enhance the SAM features with high-frequency information during Parameter-Efficient Fine-Tuning (PEFT). To convert the SAM features and coordinates into continuous segmentation output, we utilize Implicit Neural Representation (INR) to learn an implicit segmentation decoder. We also propose an uncertainty-guided sampling strategy for efficient learning of INR. Extensive evaluations on 2D medical image segmentation tasks have shown that our proposed method with only 1.6M trainable parameters outperforms existing methods including discrete and implicit methods. The code will be available at: https://github.com/ucwxb/I-MedSAM.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (43)
  1. Mohammad D Alahmadi. Boundary aware u-net for medical image segmentation. Arabian Journal for Science and Engineering, 48(8):9929–9940, 2023.
  2. WM-DOVA maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians. Computerized Medical Imaging and Graphics, 43:99–111, 2015.
  3. Transunet: Transformers make strong encoders for medical image segmentation. arXiv preprint arXiv:2102.04306, 2021.
  4. Implicit functions in feature space for 3d shape reconstruction and completion. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6970–6981, 2020.
  5. PraNet: Parallel reverse attention network for polyp segmentation. In MICCAI, pages 263–273. Springer, 2020.
  6. Diffdp: Radiotherapy dose prediction via a diffusion model. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 191–201. Springer, 2023.
  7. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. international conference on machine learning, 2015.
  8. Res2Net: A new multi-scale backbone architecture. IEEE TPAMI, 43(2):652–662, 2019.
  9. On calibration of modern neural networks. international conference on machine learning, 2017.
  10. UNETR: Transformers for 3D medical image segmentation. In IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 574–584, 2022.
  11. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021.
  12. Learning implicit feature alignment function for semantic segmentation. In ECCV, pages 487–505. Springer, 2022.
  13. nnU-Net: A self-configuring method for deep learning-based biomedical image segmentation. Nature Methods, 18(2):203–211, 2021.
  14. A comprehensive study on colorectal polyp segmentation with ResUNet++, conditional random field and test-time augmentation. IEEE Journal of Biomedical and Health Informatics, 25(6):2029–2040, 2021.
  15. AMOS: A large-scale abdominal multi-organ benchmark for versatile medical image segmentation. ArXiv:2206.08023, 2022.
  16. Implicit neural representations for medical imaging segmentation. In MICCAI, 2022a.
  17. Implicit neural representations for medical imaging segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 433–443. Springer, 2022b.
  18. Segment anything. arXiv preprint arXiv:2304.02643, 2023.
  19. Miccai multi-atlas labeling beyond the cranial vault–workshop and challenge. In Proc. MICCAI Multi-Atlas Labeling Beyond Cranial Vault—Workshop Challenge, page 12, 2015.
  20. Zero-shot medical image translation via frequency-guided diffusion models. arXiv preprint arXiv:2304.02742, 2023.
  21. Decoupled weight decay regularization. In ICLR, 2017.
  22. Segment anything in medical images. arXiv preprint arXiv:2304.12306, 2023.
  23. Intriguing findings of frequency selection for image deblurring. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 1905–1913, 2023.
  24. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1):99–106, 2021.
  25. Implicit neural representation in medical imaging: A comparative survey. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2381–2391, 2023.
  26. Implicit field learning for unsupervised anomaly detection in medical images. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part II 24, pages 189–198. Springer, 2021.
  27. Anitha Pasupathy. The neural basis of image segmentation in the primate brain. Neuroscience, 296:101–109, 2015.
  28. Butterfly transform: An efficient fft based neural architecture design. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12024–12033, 2020.
  29. On the spectral bias of neural networks. In International Conference on Machine Learning, pages 5301–5310. PMLR, 2019.
  30. OSS-Net: Memory efficient high resolution semantic segmentation of 3D medical data. In British Machine Vision Conference, 2021.
  31. U-Net: Convolutional networks for biomedical image segmentation. In MICCAI, pages 234–241. Springer, 2015.
  32. NUDF: Neural unsigned distance fields for high resolution 3D medical image segmentation. ISBI, pages 1–5, 2022.
  33. Introducing frequency representation into convolution neural networks for medical image segmentation via twin-kernel fourier convolution. Computer Methods and Programs in Biomedicine, 205:106110, 2021.
  34. Attention is all you need. Advances in neural information processing systems, 30, 2017.
  35. Noc: High-quality neural object cloning with 3d lifting of segment anything. arXiv preprint arXiv:2309.12790, 2023.
  36. Medical sam adapter: Adapting segment anything model for medical image segmentation. arXiv preprint arXiv:2304.12620, 2023.
  37. Training behavior of deep neural network in frequency domain. In Neural Information Processing: 26th International Conference, ICONIP 2019, Sydney, NSW, Australia, December 12–15, 2019, Proceedings, Part I 26, pages 264–274. Springer, 2019.
  38. Ff-former: Swin fourier transformer for nighttime flare removal. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2823–2831, 2023a.
  39. Customized segment anything model for medical image segmentation. arXiv preprint arXiv:2304.13785, 2023.
  40. How segment anything model (sam) boost medical image segmentation? arXiv preprint arXiv:2305.03678, 2023.
  41. Keep your friends close & enemies farther: Debiasing contrastive learning with spatial priors in 3D radiology images. In 2022 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pages 1824–1829. IEEE, 2022.
  42. Swipe: Efficient and robust medical image segmentation with implicit patch embeddings. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 315–326. Springer, 2023b.
  43. Attention retractable frequency fusion transformer for image super resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1756–1763, 2023.
Citations (6)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now