Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

3D-MIR: A Benchmark and Empirical Study on 3D Medical Image Retrieval in Radiology (2311.13752v1)

Published 23 Nov 2023 in cs.CV and cs.AI

Abstract: The increasing use of medical imaging in healthcare settings presents a significant challenge due to the increasing workload for radiologists, yet it also offers opportunity for enhancing healthcare outcomes if effectively leveraged. 3D image retrieval holds potential to reduce radiologist workloads by enabling clinicians to efficiently search through diagnostically similar or otherwise relevant cases, resulting in faster and more precise diagnoses. However, the field of 3D medical image retrieval is still emerging, lacking established evaluation benchmarks, comprehensive datasets, and thorough studies. This paper attempts to bridge this gap by introducing a novel benchmark for 3D Medical Image Retrieval (3D-MIR) that encompasses four different anatomies imaged with computed tomography. Using this benchmark, we explore a diverse set of search strategies that use aggregated 2D slices, 3D volumes, and multi-modal embeddings from popular multi-modal foundation models as queries. Quantitative and qualitative assessments of each approach are provided alongside an in-depth discussion that offers insight for future research. To promote the advancement of this field, our benchmark, dataset, and code are made publicly available.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (41)
  1. Physician task load and the risk of burnout among us physicians in a national survey. The Joint Commission Journal on Quality and Patient Safety, 47(2):76–85, 2021.
  2. Shiv Ram Dubey. A decade survey of content based image retrieval using deep learning. IEEE Transactions on Circuits and Systems for Video Technology, 32(5):2687–2704, 2021.
  3. Contrastive learning of medical visual representations from paired images and text. In Machine Learning for Healthcare Conference, pages 2–25. PMLR, 2022.
  4. Effective diagnosis and treatment through content-based medical image retrieval (cbmir) by using artificial intelligence. Journal of clinical medicine, 8(4):462, 2019.
  5. Medical image retrieval using deep convolutional neural network. Neurocomputing, 266:8–20, 2017.
  6. Medfmc: A real-world dataset and benchmark for foundation model adaptation in medical image classification. arXiv preprint arXiv:2306.09579, 2023.
  7. Medmnist classification decathlon: A lightweight automl benchmark for medical image analysis. In IEEE 18th International Symposium on Biomedical Imaging (ISBI), pages 191–195, 2021.
  8. Benchmd: A benchmark for modality-agnostic learning on medical images and sensors. CoRR, abs/2304.08486, 2023.
  9. Medmnist v2 - a large-scale lightweight benchmark for 2d and 3d biomedical image classification. Scientific Data, 10(41), 2023.
  10. Radiological Society of North America. Rsna abdominal trauma detection ai challenge, 2023.
  11. Rsna brain tumor ai challenge. 2021.
  12. Rsna cervical spine fracture ai challenge. 2022.
  13. Miccai 2015 multi-atlas abdomen labeling challenge. 2015.
  14. Automatic multi-organ segmentation on abdominal ct with dense v-networks. IEEE Trans Med Imaging, 37(8):1822–1834, 2018.
  15. M. Antonelli et al. The medical segmentation decathlon. Nature Communications, 13(4128), 2022.
  16. Totalsegmentator: Robust segmentation of 104 anatomic structures in ct images. Radiology: Artificial Intelligence, 5(5), 2023.
  17. 3d ux-net: A large kernel volumetric convnet modernizing hierarchical transformer for medical image segmentation. arXiv preprint arXiv:2209.15076, 2022.
  18. Scaling up 3d kernels with bayesian frequency re-parameterization for medical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 632–641. Springer, 2023.
  19. Immediate roi search for 3d medical images. In MICCAI workshop on Medical Content-based Retrieval for Clinical Decision Support (MCBR-CDS), October 2012.
  20. A comprehensive review of content-based image retrieval systems using deep learning and hand-crafted features in medical imaging: Research challenges and future directions. Computers and Electrical Engineering, 104:108450, 2022.
  21. Large-scale retrieval for medical image analytics: A comprehensive review. Med Image Anal., (43):66–84, 2018.
  22. Billion-scale similarity search with GPUs. IEEE Transactions on Big Data, 7(3):535–547, 2019.
  23. Content based image retrieval. Int. J. Adv. Eng. Glob. Technol, 3:1251–1258, 2015.
  24. A survey on biomedical image captioning. In Raffaella Bernardi, Raquel Fernandez, Spandana Gella, Kushal Kafle, Christopher Kanan, Stefan Lee, and Moin Nabi, editors, Proceedings of the Second Workshop on Shortcomings in Vision and Language, pages 26–36, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics.
  25. Overview of the imageclefmed 2021 concept & caption prediction task. In Guglielmo Faggioli, Nicola Ferro, Alexis Joly, Maria Maistro, and Florina Piroi, editors, Proceedings of the Working Notes of CLEF 2021 - Conference and Labs of the Evaluation Forum, Bucharest, Romania, September 21st - to - 24th, 2021, volume 2936 of CEUR Workshop Proceedings, pages 1101–1112. CEUR-WS.org, 2021.
  26. Overview of imageclefmedical 2023 - caption prediction and concept detection. In Mohammad Aliannejadi, Guglielmo Faggioli, Nicola Ferro, and Michalis Vlachos, editors, Working Notes of the Conference and Labs of the Evaluation Forum (CLEF 2023), Thessaloniki, Greece, September 18th to 21st, 2023, volume 3497 of CEUR Workshop Proceedings, pages 1328–1346. CEUR-WS.org, 2023.
  27. Medical image captioning via generative pretrained transformers. Scientific Reports, 13, 2023.
  28. Learning transferable visual models from natural language supervision. arXiv 2103.00020, 2021.
  29. Y. Lu et. al. Florence: A new foundation model for computer vision. arXiv 2111.11432v1, 2021.
  30. Large-scale domain-specific pretraining for biomedical vision-language processing. CoRR, abs/2303.00915, 2023.
  31. Region-based contrastive pretraining for medical image retrieval with anatomic query. arXiv preprint arXiv:2305.05598, 2023.
  32. Sean Berryman. Pancreas-ct. The Cancer Imaging Archive (TCIA) Public Access, 2023.
  33. StanfordMIMI. Stanford knee mri multi-task evaluation (skm-tea) dataset. GitHub, 2023.
  34. Rad-chestct dataset. CVIT - Center for Virtual Imaging Trials, 2023.
  35. A large, curated, open-source stroke neuroimaging dataset to advance machine learning in clinical research. Nature Scientific Data, 9(1):1–10, 2022.
  36. Episurg: An epilepsy surgery mri dataset. Data in brief, 32:106231, 2020.
  37. nnu-net: a self-configuring method for deep learning-based biomedical image segmentation. Nature methods, 18(2), 2021.
  38. OpenAI. Gpt-4 technical report. ArXiv, abs/2303.08774, 2023.
  39. Automatic multi-organ segmentation on abdominal ct with dense v-networks. IEEE Transactions on Medical Imaging, 37(8):1822–1834, 2018.
  40. AJCC Cancer Staging Manual. Springer, 2017.
  41. The insight segmentation and registration toolkit. Software Guide, Updated for ITK version 5.2.0, 2021.
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Github Logo Streamline Icon: https://streamlinehq.com

GitHub

  1. GitHub - abachaa/3D-MIR: 3D Medical Image Retrieval in Radiology (16 stars)