Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
149 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Clustering Propagation for Universal Medical Image Segmentation (2403.16646v1)

Published 25 Mar 2024 in cs.CV

Abstract: Prominent solutions for medical image segmentation are typically tailored for automatic or interactive setups, posing challenges in facilitating progress achieved in one task to another.${!}$ This${!}$ also${!}$ necessitates${!}$ separate${!}$ models for each task, duplicating both training time and parameters.${!}$ To${!}$ address${!}$ above${!}$ issues,${!}$ we${!}$ introduce${!}$ S2VNet,${!}$ a${!}$ universal${!}$ framework${!}$ that${!}$ leverages${!}$ Slice-to-Volume${!}$ propagation${!}$ to${!}$ unify automatic/interactive segmentation within a single model and one training session. Inspired by clustering-based segmentation techniques, S2VNet makes full use of the slice-wise structure of volumetric data by initializing cluster centers from the cluster${!}$ results${!}$ of${!}$ previous${!}$ slice.${!}$ This enables knowledge acquired from prior slices to assist in the segmentation of the current slice, further efficiently bridging the communication between remote slices using mere 2D networks. Moreover, such a framework readily accommodates interactive segmentation with no architectural change, simply by initializing centroids from user inputs. S2VNet distinguishes itself by swift inference speeds and reduced memory consumption compared to prevailing 3D solutions. It can also handle multi-class interactions with each of them serving to initialize different centroids. Experiments on three benchmarks demonstrate S2VNet surpasses task-specified solutions on both automatic/interactive setups.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (117)
  1. Medical image segmentation using deep learning: A survey. IET Image Processing, 16(5):1243–1267, 2022.
  2. Current methods in medical image segmentation. Annual Review of Biomedical Engineering, 2(1):315–337, 2000.
  3. Interactive medical image segmentation using deep learning with image-specific fine tuning. IEEE TMI, 37(7):1562–1573, 2018.
  4. Volumetric memory network for interactive medical image segmentation. Medical Image Analysis, 83:102599, 2023.
  5. U-net: Convolutional networks for biomedical image segmentation. In MICCAI, 2015.
  6. V-net: Fully convolutional neural networks for volumetric medical image segmentation. In IEEE 3DV, 2016.
  7. Unext: Mlp-based rapid medical image segmentation network. In MICCAI, 2022.
  8. 3d u-net: learning dense volumetric segmentation from sparse annotation. In MICCAI, 2016.
  9. Efficient multi-scale 3d cnn with fully connected crf for accurate brain lesion segmentation. Medical Image Analysis, 36:61–78, 2017.
  10. Unet++: Redesigning skip connections to exploit multiscale features in image segmentation. IEEE TMI, 39(6):1856–1867, 2019.
  11. Msrf-net: a multi-scale residual fusion network for biomedical image segmentation. IEEE Journal of Biomedical and Health Informatics, 26(5):2252–2263, 2021.
  12. Abdominal multi-organ segmentation with organ-attention networks and statistical fusion. Medical Image Analysis, 55:88–102, 2019.
  13. Hierarchical attention networks for medical image segmentation. arXiv preprint arXiv:1911.08777, 2019.
  14. Volumetric attention for 3d medical image segmentation and detection. In MICCAI, 2019.
  15. Deepigeos: a deep interactive geodesic framework for medical image segmentation. IEEE TPAMI, 41(7):1559–1572, 2018.
  16. Mideepseg: Minimally interactive segmentation of unseen objects from medical images using deep learning. Medical Image Analysis, 72:102102, 2021.
  17. Instance segmentation by jointly optimizing spatial embeddings and clustering bandwidth. In CVPR, 2019.
  18. Cmt-deeplab: Clustering mask transformers for panoptic segmentation. In CVPR, 2022.
  19. k-means mask transformer. In ECCV, 2022.
  20. Clustseg: Clustering for universal segmentation. In ICML, 2023.
  21. Transforming the interactive segmentation for medical imaging. In MICCAI, 2022.
  22. Quality-aware memory network for interactive volumetric image segmentation. In MICCAI, 2021.
  23. Word: A large scale dataset, benchmark and clinical applicable study for abdominal organ segmentation from ct image. Medical Image Analysis, 82:102642, 2022.
  24. Miccai multi-atlas labeling beyond the cranial vault–workshop and challenge. In MICCAI Multi-Atlas Labeling Beyond Cranial Vault—Workshop Challenge, 2015.
  25. Amos: A large-scale abdominal multi-organ benchmark for versatile medical image segmentation. In NeurIPS, 2022.
  26. Rfnet: Region-aware fusion network for incomplete multi-modal brain tumor segmentation. In CVPR, 2021.
  27. Modeling the probabilistic distribution of unlabeled data for one-shot medical image segmentation. In AAAI, 2021.
  28. Genesegnet: a deep learning framework for cell segmentation by integrating gene expression and imaging. Genome Biology, 24(1):235, 2023.
  29. A 3d coarse-to-fine framework for volumetric medical image segmentation. In IEEE 3DV, 2018.
  30. Capsules for biomedical image segmentation. Medical Image Analysis, 68:101889, 2021.
  31. Mixed-supervised dual-network for medical image segmentation. In MICCAI, 2019.
  32. Quantization of fully convolutional networks for accurate biomedical image segmentation. In CVPR, 2018.
  33. Graph flow: Cross-layer graph flow distillation for dual efficient medical image segmentation. IEEE TMI, 42(4):1159–1171, 2022.
  34. Unet++: A nested u-net architecture for medical image segmentation. In MICCAI Workshop, 2018.
  35. Unet 3+: A full-scale connected unet for medical image segmentation. In IEEE ICASSP, 2020.
  36. High-resolution encoder–decoder networks for low-contrast medical image segmentation. IEEE TIP, 29:461–475, 2019.
  37. Ce-net: Context encoder network for 2d medical image segmentation. IEEE TMI, 38(10):2281–2292, 2019.
  38. Kiu-net: Overcomplete convolutional architectures for biomedical image and volumetric segmentation. IEEE TMI, 41(4):965–976, 2021.
  39. Segmentation ability map: Interpret deep features for medical image segmentation. Medical Image Analysis, 84:102726, 2023.
  40. Ace-net: biomedical image segmentation with augmented contracting and expansive paths. In MICCAI, 2019.
  41. Structure boundary preserving segmentation for medical image with ambiguous boundary. In CVPR, 2020.
  42. Et-net: A generic edge-attention guidance network for medical image segmentation. In MICCAI, 2019.
  43. Ca-net: Comprehensive attention convolutional neural networks for explainable medical image segmentation. IEEE TMI, 40(2):699–711, 2020.
  44. A spatiotemporal volumetric interpolation network for 4d dynamic medical image. In CVPR, 2020.
  45. Bix-nas: Searching efficient bi-directional architecture for medical image segmentation. In MICCAI, 2021.
  46. Ms-nas: Multi-scale neural architecture search for medical image segmentation. In MICCAI, 2020.
  47. Uxnet: Searching multi-level feature aggregation for 3d medical image segmentation. In MICCAI, 2020.
  48. C2fnas: Coarse-to-fine neural architecture search for 3d medical image segmentation. In CVPR, 2020.
  49. nnu-net: a self-configuring method for deep learning-based biomedical image segmentation. Nature methods, 18(2):203–211, 2021.
  50. Transunet: Transformers make strong encoders for medical image segmentation. arXiv preprint arXiv:2102.04306, 2021.
  51. Medical transformer: Gated axial-attention for medical image segmentation. In MICCAI, 2021.
  52. Utnet: a hybrid transformer architecture for medical image segmentation. In MICCAI, 2021.
  53. Unetr: Transformers for 3d medical image segmentation. In WACV, 2022.
  54. Cotr: Efficiently bridging cnn and transformer for 3d medical image segmentation. In MICCAI, 2021.
  55. Swin unetr: Swin transformers for semantic segmentation of brain tumors in mri images. In International MICCAI Brainlesion Workshop, 2021.
  56. Swinmm: masked multi-view with swin transformers for 3d medical image segmentation. In MICCAI, 2023.
  57. Transbts: Multimodal brain tumor segmentation using transformer. In MICCAI, 2021.
  58. nnformer: Interleaved transformer for volumetric segmentation. arXiv preprint arXiv:2109.03201, 2021.
  59. Instanceformer: An online video instance segmentation framework. In AAAI, volume 37, pages 1188–1195, 2023.
  60. A generalized framework for video instance segmentation. In CVPR, pages 14623–14632, 2023.
  61. Semi-supervised video object segmentation with super-trajectories. IEEE TPAMI, 41(4):985–998, 2018.
  62. Locality-aware inter-and intra-video reconstruction for self-supervised correspondence learning. In CVPR, 2022.
  63. Unified mask embedding and correspondence learning for self-supervised video segmentation. In CVPR, 2023.
  64. Boosting video object segmentation via space-time correspondence learning. In CVPR, 2023.
  65. Adversarial style mining for one-shot unsupervised domain adaptation. NeurIPS, 33:20612–20623, 2020.
  66. Taking a closer look at domain shift: Category-level adversaries for semantics consistent domain adaptation. In CVPR, pages 2507–2516, 2019.
  67. Category-level adversarial adaptation for semantic segmentation using purified features. IEEE TPAMI, 44(8):3940–3956, 2021.
  68. A survey on active learning and human-in-the-loop deep learning for medical image analysis. Medical Image Analysis, 71:102062, 2021.
  69. Interactive graph cuts for optimal boundary & region segmentation of objects in nd images. In ICCV, 2001.
  70. Slic-seg: A minimally interactive segmentation of the placenta from sparse and motion-corrupted fetal mri in multiple views. Medical Image Analysis, 34:137–147, 2016.
  71. Interactive few-shot learning: Limited supervision, better medical image segmentation. IEEE TMI, 40(10):2575–2588, 2021.
  72. Uncertainty-guided efficient interactive refinement of fetal brain segmentation from stacks of mri slices. In MICCAI, 2020.
  73. Ilastik: interactive machine learning for (bio) image analysis. Nature Methods, 16(12):1226–1232, 2019.
  74. Deepcut: Object segmentation from bounding box annotations using convolutional neural networks. IEEE TMI, 36(2):674–683, 2016.
  75. A fixed-point model for pancreas segmentation in abdominal ct scans. In MICCAI, 2017.
  76. Scribble2label: Scribble-supervised cell segmentation via self-generating pseudo-labels with consistency. In MICCAI, 2020.
  77. Nuclick: a deep learning framework for interactive segmentation of microscopic images. Medical Image Analysis, 65:101771, 2020.
  78. Extreme points derived confidence map as a cue for class-agnostic interactive segmentation using deep neural network. In MICCAI, 2019.
  79. Guiding the guidance: A comparative analysis of user guidance signals for interactive segmentation of volumetric images. arXiv preprint arXiv:2303.06942, 2023.
  80. Efficient and generic interactive segmentation framework to correct mispredictions during clinical evaluation of medical images. In MICCAI, 2021.
  81. isegformer: Interactive segmentation via transformers with application to 3d knee mr images. In MICCAI, 2022.
  82. A hybrid propagation network for interactive volumetric image segmentation. In MICCAI, 2022.
  83. Exploring cycle consistency learning in interactive volume segmentation. arXiv preprint arXiv:2303.06493, 2023.
  84. Image segmentation using k-means clustering algorithm and subtractive clustering algorithm. Procedia Computer Science, 54:764–771, 2015.
  85. K-means cluster analysis for image segmentation. International Journal of Computer Applications, 96(4), 2014.
  86. Determination of number of clusters in k-means clustering and application in colour image segmentation. In International Conference on Advances in Pattern Recognition and Digital Techniques, 1999.
  87. Fully convolutional networks for semantic segmentation. In CVPR, 2015.
  88. Masked-attention mask transformer for universal image segmentation. In CVPR, 2022.
  89. Gmmseg: Gaussian mixture based generative semantic segmentation models. NeurIPS, 35:31360–31375, 2022.
  90. Rethinking semantic segmentation: A prototype view. In CVPR, pages 2582–2593, 2022.
  91. Exploring cross-image pixel contrast for semantic segmentation. In ICCV, pages 7303–7313, 2021.
  92. Deep hierarchical semantic segmentation. In CVPR, pages 1246–1257, 2022.
  93. Logic-induced diagnostic reasoning for semi-supervised semantic segmentation. In ICCV, 2023.
  94. Logicseg: Parsing visual semantics with neural logic learning and reasoning. In ICCV, 2023.
  95. Segment and track anything. arXiv preprint arXiv:2305.06558, 2023.
  96. Semantic hierarchy-aware segmentation. IEEE TPAMI, 2023.
  97. Omg-seg: Is one model good enough for all segmentation? In CVPR, 2024.
  98. Recurrent pixel embedding for instance grouping. In CVPR, 2018.
  99. Clustering based point cloud representation learning for 3d analysis. In ICCV, 2023.
  100. Proposalcontrast: Unsupervised pre-training for lidar-based 3d object detection. In ECCV, 2022.
  101. Rethinking visual feature extraction: Modeling representatives from a neural clustering view. In CVPR, 2024.
  102. End-to-end object detection with transformers. In ECCV, 2020.
  103. Per-pixel classification is not all you need for semantic segmentation. In NeurIPS, 2021.
  104. Stuart Lloyd. Least squares quantization in pcm. IEEE Transactions on Information Theory, 28(2):129–137, 1982.
  105. A brain tumor segmentation framework based on outlier detection. Medical Image Analysis, 8(3):275–283, 2004.
  106. Swin transformer: Hierarchical vision transformer using shifted windows. In ICCV, 2021.
  107. Deep interactive object selection. In CVPR, 2016.
  108. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  109. Imagenet: A large-scale hierarchical image database. In CVPR, 2009.
  110. The medical segmentation decathlon. Nature communications, 13(1):4128, 2022.
  111. Lee R Dice. Measures of the amount of ecologic association between species. Ecology, 26(3):297–302, 1945.
  112. Comparing images using the hausdorff distance. IEEE TPAMI, 15(9):850–863, 1993.
  113. Iteratively-refined interactive 3d medical image segmentation with multi-agent reinforcement learning. In CVPR, 2020.
  114. Espnet: Efficient spatial pyramid of dilated convolutions for semantic segmentation. In ECCV, 2018.
  115. 3d dilated multi-fiber network for real-time brain tumor segmentation in mri. In MICCAI, 2019.
  116. Lcov-net: A lightweight neural network for covid-19 pneumonia lesion segmentation from 3d ct images. In International Symposium on Biomedical Imaging, 2021.
  117. 3d ux-net: A large kernel volumetric convnet modernizing hierarchical transformer for medical image segmentation. In ICLR, 2023.
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now