Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Uncertainty Quantification in Detecting Choroidal Metastases on MRI via Evolutionary Strategies (2404.08853v1)

Published 12 Apr 2024 in cs.CV, cs.LG, and cs.NE

Abstract: Uncertainty quantification plays a vital role in facilitating the practical implementation of AI in radiology by addressing growing concerns around trustworthiness. Given the challenges associated with acquiring large, annotated datasets in this field, there is a need for methods that enable uncertainty quantification in small data AI approaches tailored to radiology images. In this study, we focused on uncertainty quantification within the context of the small data evolutionary strategies-based technique of deep neuroevolution (DNE). Specifically, we employed DNE to train a simple Convolutional Neural Network (CNN) with MRI images of the eyes for binary classification. The goal was to distinguish between normal eyes and those with metastatic tumors called choroidal metastases. The training set comprised 18 images with choroidal metastases and 18 without tumors, while the testing set contained a tumor-to-normal ratio of 15:15. We trained CNN model weights via DNE for approximately 40,000 episodes, ultimately reaching a convergence of 100% accuracy on the training set. We saved all models that achieved maximal training set accuracy. Then, by applying these models to the testing set, we established an ensemble method for uncertainty quantification.The saved set of models produced distributions for each testing set image between the two classes of normal and tumor-containing. The relative frequencies permitted uncertainty quantification of model predictions. Intriguingly, we found that subjective features appreciated by human radiologists explained images for which uncertainty was high, highlighting the significance of uncertainty quantification in AI-driven radiological analyses.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (30)
  1. Hercules: Deep hierarchical attentive multilevel fusion model with uncertainty quantification for medical image classification. IEEE Transactions on Industrial Informatics, 19(1): 274–285.
  2. A review of uncertainty quantification in deep learning: Techniques, applications and challenges. Information fusion, 76: 243–297.
  3. A novel deep neuroevolution-based image classification method to diagnose coronavirus disease (COVID-19). Computers in biology and medicine, 139: 104994.
  4. Hyperparameter optimization: Comparing genetic algorithm against grid search and bayesian optimization. In 2021 IEEE Congress on Evolutionary Computation (CEC), 1551–1559. IEEE.
  5. Fast denser: Efficient deep neuroevolution. In european conference on genetic programming, 197–212. Springer.
  6. Random search for hyper-parameter optimization. Journal of machine learning research, 13(2).
  7. Google colaboratory. Building machine learning and deep learning models on google cloud platform: a comprehensive guide for beginners, 59–64.
  8. Cifci, M. A. 2023. A Deep Learning-Based Framework for Uncertainty Quantification in Medical Imaging Using the DropWeak Technique: An Empirical Study with Baresnet. Diagnostics, 13(4): 800.
  9. Mitigating bias in radiology machine learning: 3. Performance metrics. Radiology: Artificial Intelligence, 4(5): e220061.
  10. Quantifying Uncertainty in Deep Learning of Radiologic Images. Radiology, 308(2): e222217.
  11. On the effects of quantisation on model uncertainty in bayesian neural networks. In Uncertainty in Artificial Intelligence, 929–938. PMLR.
  12. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning, 1050–1059. PMLR.
  13. What uncertainties do we need in bayesian deep learning for computer vision? Advances in neural information processing systems, 30.
  14. Automatic differentiation in deep learning. Deep Learning with Python: Learn Best Practices of Deep Learning Models with PyTorch, 133–145.
  15. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in neural information processing systems, 30.
  16. Trustworthy clinical AI solutions: a unified review of uncertainty quantification in deep learning models for medical image analysis. arXiv preprint arXiv:2210.03736.
  17. A radiology-focused review of predictive uncertainty for AI interpretability in computer-assisted segmentation. Radiology: Artificial Intelligence, 3(6): e210031.
  18. How sure are you, doctor? A standardized lexicon to describe the radiologist’s level of certainty. American Journal of Roentgenology, 207(1): 2–3.
  19. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32.
  20. Deep neuroevolution for limited, heterogeneous data: proof-of-concept application to Neuroblastoma brain metastasis using a small virtual pooled image collection. arXiv preprint arXiv:2211.14499.
  21. Early exit ensembles for uncertainty quantification. In Machine Learning for Health, 181–195. PMLR.
  22. Uncertainty quantification in medical image segmentation with normalizing flows. In Machine Learning in Medical Imaging: 11th International Workshop, MLMI 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4, 2020, Proceedings 11, 80–90. Springer.
  23. Spall, J. C. 1992. Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE transactions on automatic control, 37(3): 332–341.
  24. Deep neuroevolution to predict astrocytoma grade from functional brain networks. In 2023 5th International Conference on Intelligent Medicine and Image Processing (IMIP), 1–6. IEEE.
  25. Deep Neuroevolution Squeezes More out of Small Neural Networks and Small Training Sets: Sample Application to MRI Brain Sequence Classification. In International Symposium on Intelligent Informatics, 153–167. Springer.
  26. Direct Evaluation of Treatment Response in Brain Metastatic Disease with Deep Neuroevolution. Journal of Digital Imaging, 36(2): 536–546.
  27. Optimising hardware accelerated neural networks with quantisation and a knowledge distillation evolutionary algorithm. Electronics, 10(4): 396.
  28. Deep neuroevolution: Genetic algorithms are a competitive alternative for training deep neural networks for reinforcement learning. arXiv preprint arXiv:1712.06567.
  29. Uncertainty quantification and estimation in medical image classification. In Artificial Neural Networks and Machine Learning–ICANN 2021: 30th International Conference on Artificial Neural Networks, Bratislava, Slovakia, September 14–17, 2021, Proceedings, Part III 30, 671–683. Springer.
  30. Sample Efficient Deep Neuroevolution in Low Dimensional Latent Space.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Bala McRae-Posani (1 paper)
  2. Andrei Holodny (5 papers)
  3. Hrithwik Shalu (20 papers)
  4. Joseph N Stember (3 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now