Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

TriSAM: Tri-Plane SAM for zero-shot cortical blood vessel segmentation in VEM images (2401.13961v4)

Published 25 Jan 2024 in cs.CV

Abstract: While imaging techniques at macro and mesoscales have garnered substantial attention and resources, microscale Volume Electron Microscopy (vEM) imaging, capable of revealing intricate vascular details, has lacked the necessary benchmarking infrastructure. In this paper, we address a significant gap in this field of neuroimaging by introducing the first-in-class public benchmark, BvEM, designed specifically for cortical blood vessel segmentation in vEM images. Our BvEM benchmark is based on vEM image volumes from three mammals: adult mouse, macaque, and human. We standardized the resolution, addressed imaging variations, and meticulously annotated blood vessels through semi-automatic, manual, and quality control processes, ensuring high-quality 3D segmentation. Furthermore, we developed a zero-shot cortical blood vessel segmentation method named TriSAM, which leverages the powerful segmentation model SAM for 3D segmentation. To extend SAM from 2D to 3D volume segmentation, TriSAM employs a multi-seed tracking framework, leveraging the reliability of certain image planes for tracking while using others to identify potential turning points. This approach effectively achieves long-term 3D blood vessel segmentation without model training or fine-tuning. Experimental results show that TriSAM achieved superior performances on the BvEM benchmark across three species. Our dataset, code, and model are available online at \url{https://jia-wan.github.io/bvem}.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (45)
  1. Segment anything for microscopy. bioRxiv, pp.  2023–08, 2023a.
  2. Segment anything for microscopy. bioRxiv, 2023b. URL https://api.semanticscholar.org/CorpusID:261125076.
  3. Vast (volume annotation and segmentation tool): efficient manual and semi-automatic labeling of large 3d image stacks. Frontiers in neural circuits, 12:88, 2018.
  4. Automatic detection of synaptic partners in a whole-brain drosophila electron microscopy data set. Nature methods, 18(7):771–774, 2021.
  5. Sam fails to segment anything?–sam-adapter: Adapting sam in underperformed scenes: Camouflage, shadow, and more. arXiv preprint arXiv:2304.09148, 2023.
  6. Sam-med2d. arXiv preprint arXiv:2308.16184, 2023a.
  7. Segment and track anything. arXiv preprint arXiv:2305.06558, 2023b.
  8. 3d u-net: learning dense volumetric segmentation from sparse annotation. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2016: 19th International Conference, Athens, Greece, October 17-21, 2016, Proceedings, Part II 19, pp.  424–432. Springer, 2016.
  9. Vessel-captcha: an efficient learning framework for vessel annotation and segmentation. Medical Image Analysis, 75:102263, 2022.
  10. Segment anything model (sam) for digital pathology: Assess zero-shot segmentation on whole slide imaging. arXiv preprint arXiv:2304.04155, 2023.
  11. Lee R Dice. Measures of the amount of ecologic association between species. Ecology, 26(3):297–302, 1945.
  12. Neuronal wiring diagram of an adult brain. bioRxiv, 2023. doi: 10.1101/2023.06.27.546656. URL https://www.biorxiv.org/content/early/2023/07/11/2023.06.27.546656.
  13. Quantifying mesoscale neuroanatomy using x-ray microtomography. eneuro, 4(5), 2017.
  14. Editanything: Empowering unparalleled flexibility in image editing and generation. In Proceedings of the 31st ACM International Conference on Multimedia, Demo track, 2023.
  15. Brain vessel segmentation using deep learning-a review. IEEE Access, 2022.
  16. Raj N Kalaria. Vascular basis for brain degeneration: faltering controls and risk factors for dementia. Nutrition reviews, 68(suppl_2):S74–S87, 2010.
  17. Segment anything. arXiv preprint arXiv:2304.02643, 2023.
  18. Convolutional nets for reconstructing neural circuits from brain images acquired by serial section electron microscopy. Current opinion in neurobiology, 55:188–198, 2019.
  19. Segment any point cloud sequences by distilling vision foundation models. arXiv preprint arXiv:2306.09347, 2023.
  20. A u-net deep learning framework for high performance vessel segmentation in patients with cerebrovascular disease. Frontiers in neuroscience, 13:97, 2019.
  21. High resolution multiplex confocal imaging of the neurovascular unit in health and experimental ischemic stroke. Cells, 12(4):645, 2023.
  22. Connectomic comparison of mouse and human cortex. Science, 377(6602):eabo0924, 2022.
  23. Can sam boost video super-resolution? arXiv preprint arXiv:2305.06524, 2023.
  24. Jun Ma and Bo Wang. Segment anything in medical images. arXiv preprint arXiv:2304.12306, 2023.
  25. Segment anything model for medical image analysis: an experimental study. Medical Image Analysis, 89:102918, 2023.
  26. Imaging of angiogenesis: from microscope to clinic. Nature medicine, 9(6):713–725, 2003.
  27. 3d segmentation in the clinic: A grand challenge ii-coronary artery tracking. Insight Journal, 1(5):6, 2008.
  28. Blood vessel segmentation algorithms—review of methods, datasets and evaluation metrics. Computer methods and programs in biomedicine, 158:71–91, 2018.
  29. Exploring the third dimension: volume electron microscopy comes of age. Micron, 61:9–19, 2014.
  30. Volume electron microscopy. Nature Reviews Methods Primers, 2(1):51, 2022.
  31. Jeffrey L Saver. Time is brain—quantified. Stroke, 37(1):263–266, 2006.
  32. A connectomic study of a petascale fragment of human cerebral cortex. BioRxiv, pp.  2021–05, 2021.
  33. Kimimaro: Skeletonize densely labeled 3d image segmentations. Zenodo https://doi. org/10.5281/zenodo, 5539912, 2021.
  34. Deepvesselnet: Vessel segmentation, centerline prediction, and bifurcation detection in 3-d angiographic volumes. Frontiers in Neuroscience, 14:1285, 2020.
  35. Functional connectomics spanning multiple areas of mouse visual cortex. bioRxiv, 2023. doi: 10.1101/2021.07.28.454025. URL https://www.biorxiv.org/content/early/2023/04/19/2021.07.28.454025.
  36. Synaptic partner assignment using attentional voxel association networks. In 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), pp.  1–5. IEEE, 2020.
  37. Star-convex polyhedra for 3d object detection and segmentation in microscopy. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp.  3666–3673, 2020.
  38. Accurate and versatile 3d segmentation of plant tissues at cellular resolution. eLife, 9:e57613, jul 2020. ISSN 2050-084X. doi: 10.7554/eLife.57613. URL https://doi.org/10.7554/eLife.57613.
  39. An out-of-domain synapse detection challenge for microwasp brain connectomes. arXiv preprint arXiv:2302.00545, 2023a.
  40. Medical sam adapter: Adapting segment anything model for medical image segmentation. arXiv preprint arXiv:2304.12620, 2023b.
  41. Track anything: Segment anything meets videos. arXiv preprint arXiv:2304.11968, 2023.
  42. Matte anything: Interactive natural image matting with segment anything models. arXiv preprint arXiv:2306.04121, 2023.
  43. Inpaint anything: Segment anything meets image inpainting. arXiv preprint arXiv:2304.06790, 2023.
  44. Faster segment anything: Towards lightweight sam for mobile applications. arXiv preprint arXiv:2306.14289, 2023.
  45. Fast segment anything. arXiv preprint arXiv:2306.12156, 2023.
Citations (5)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now