Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
80 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

MedFMC: A Real-world Dataset and Benchmark For Foundation Model Adaptation in Medical Image Classification (2306.09579v1)

Published 16 Jun 2023 in cs.CV

Abstract: Foundation models, often pre-trained with large-scale data, have achieved paramount success in jump-starting various vision and language applications. Recent advances further enable adapting foundation models in downstream tasks efficiently using only a few training samples, e.g., in-context learning. Yet, the application of such learning paradigms in medical image analysis remains scarce due to the shortage of publicly accessible data and benchmarks. In this paper, we aim at approaches adapting the foundation models for medical image classification and present a novel dataset and benchmark for the evaluation, i.e., examining the overall performance of accommodating the large-scale foundation models downstream on a set of diverse real-world clinical tasks. We collect five sets of medical imaging data from multiple institutes targeting a variety of real-world clinical tasks (22,349 images in total), i.e., thoracic diseases screening in X-rays, pathological lesion tissue screening, lesion detection in endoscopy images, neonatal jaundice evaluation, and diabetic retinopathy grading. Results of multiple baseline methods are demonstrated using the proposed dataset from both accuracy and cost-effective perspectives.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (30)
  1. Dosovitskiy, A. et al. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations (2021).
  2. Improving language understanding by generative pre-training. \JournalTitleOpenAI (2018).
  3. Gu, Y. et al. Domain-specific language model pretraining for biomedical natural language processing (2020). arXiv:2007.15779.
  4. Radford, A. et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, 8748–8763 (2021).
  5. Shin, H.-C. et al. Deep convolutional neural networks for computer-aided detection: Cnn architectures, dataset characteristics and transfer learning. \JournalTitleIEEE transactions on medical imaging 35, 1285–1298 (2016).
  6. Deng, J. et al. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, 248–255 (2009).
  7. Brown, T. et al. Language models are few-shot learners. \JournalTitleAdvances in neural information processing systems 33, 1877–1901 (2020).
  8. A baseline for few-shot image classification. \JournalTitlearXiv preprint arXiv:1909.02729 (2019).
  9. Rethinking few-shot image classification: a good embedding is all you need? In Proceedings of the European Conference on Computer Vision, 266–282 (Springer, 2020).
  10. Ouyang, C. et al. Self-supervision with superpixels: Training few-shot medical image segmentation without annotation. In Proceedings of the European Conference on Computer Vision, 762–780 (Springer, 2020).
  11. Singh, R. et al. Metamed: Few-shot medical image classification using gradient-based meta-learning. \JournalTitlePattern Recognition 120, 108111 (2021).
  12. Learning to prompt for vision-language models. \JournalTitleInternational Journal of Computer Vision 130, 2337–2348 (2022).
  13. Conditional prompt learning for vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 16816–16825 (2022).
  14. Caron, M. et al. Emerging properties in self-supervised vision transformers. In Proceedings of the International Conference on Computer Vision (ICCV) (2021).
  15. He, K. et al. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 16000–16009 (2022).
  16. Jia, M. et al. Visual prompt tuning. \JournalTitlearXiv preprint arXiv:2203.12119 (2022).
  17. Medical image understanding with pretrained vision language models: A comprehensive study. \JournalTitleArXiv abs/2209.15517 (2022).
  18. Sun, L. et al. Few-shot medical image segmentation using a global correlation network with discriminative embedding (2020). 2012.05440.
  19. Shakeri, F. et al. Fhist: A benchmark for few-shot classification of histological images (2022). 2206.00092.
  20. Wang, X. et al. Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. \JournalTitle2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 3462–3471 (2017).
  21. Irvin, J. A. et al. Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. \JournalTitleArXiv abs/1901.07031 (2019).
  22. Johnson, A. E. W. et al. Mimic-cxr: A large publicly available database of labeled chest radiographs. \JournalTitleArXiv abs/1901.07042 (2019).
  23. Antonelli, M. et al. The medical segmentation decathlon. \JournalTitleNature Communications 13 (2021).
  24. The ham10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. \JournalTitleScientific Data 5 (2018).
  25. the RSNA MIRC project. Dicom anonymizer. urlhttp://mirc.rsna.org/download.
  26. Da, Q. et al. Digestpath: A benchmark dataset with challenge review for the pathological detection and segmentation of digestive-system. \JournalTitleMedical Image Analysis 80, 102485 (2022).
  27. Meta-baseline: Exploring simple meta-learning for few-shot learning. \JournalTitle2021 IEEE/CVF International Conference on Computer Vision (ICCV) 9042–9051 (2020).
  28. Xie, Z. et al. Simmim: A simple framework for masked image modeling. In International Conference on Computer Vision and Pattern Recognition (CVPR) (2022).
  29. Mmclassification. https://github.com/open-mmlab/mmclassification.
  30. Wang, D. et al. A real-world dataset and benchmark for foundation1 model adaptation in medical image classification. figureshare https://doi.org/XXXXX (2023).
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (15)
  1. Dequan Wang (37 papers)
  2. Xiaosong Wang (42 papers)
  3. Lilong Wang (4 papers)
  4. Mengzhang Li (4 papers)
  5. Qian Da (2 papers)
  6. Xiaoqiang Liu (17 papers)
  7. Xiangyu Gao (31 papers)
  8. Jun Shen (63 papers)
  9. Junjun He (77 papers)
  10. Tian Shen (8 papers)
  11. Qi Duan (11 papers)
  12. Jie Zhao (214 papers)
  13. Kang Li (207 papers)
  14. Yu Qiao (563 papers)
  15. Shaoting Zhang (133 papers)
Citations (18)
View on Semantic Scholar