Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

ATOMMIC: An Advanced Toolbox for Multitask Medical Imaging Consistency to facilitate Artificial Intelligence applications from acquisition to analysis in Magnetic Resonance Imaging (2404.19665v1)

Published 30 Apr 2024 in physics.med-ph, cs.AI, cs.SE, math-ph, and math.MP

Abstract: AI is revolutionizing MRI along the acquisition and processing chain. Advanced AI frameworks have been developed to apply AI in various successive tasks, such as image reconstruction, quantitative parameter map estimation, and image segmentation. Existing frameworks are often designed to perform tasks independently or are focused on specific models or datasets, limiting generalization. We introduce ATOMMIC, an open-source toolbox that streamlines AI applications for accelerated MRI reconstruction and analysis. ATOMMIC implements several tasks using DL networks and enables MultiTask Learning (MTL) to perform related tasks integrated, targeting generalization in the MRI domain. We first review the current state of AI frameworks for MRI through a comprehensive literature search and by parsing 12,479 GitHub repositories. We benchmark 25 DL models on eight publicly available datasets to present distinct applications of ATOMMIC on accelerated MRI reconstruction, image segmentation, quantitative parameter map estimation, and joint accelerated MRI reconstruction and image segmentation utilizing MTL. Our findings demonstrate that ATOMMIC is the only MTL framework with harmonized complex-valued and real-valued data support. Evaluations on single tasks show that physics-based models, which enforce data consistency by leveraging the physical properties of MRI, outperform other models in reconstructing highly accelerated acquisitions. Physics-based models that produce high reconstruction quality can accurately estimate quantitative parameter maps. When high-performing reconstruction models are combined with robust segmentation networks utilizing MTL, performance is improved in both tasks. ATOMMIC facilitates MRI reconstruction and analysis by standardizing workflows, enhancing data interoperability, integrating unique features like MTL, and effectively benchmarking DL models.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (50)
  1. nnU-Net: A self-configuring method for deep learning-based biomedical image segmentation. Nature Methods, 18(2):203–211, February 2021. ISSN 1548-7105. doi:10.1038/s41592-020-01008-z.
  2. V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation. In 2016 Fourth International Conference on 3D Vision (3DV), pages 565–571, October 2016. doi:10.1109/3DV.2016.79.
  3. Attention U-Net: Learning Where to Look for the Pancreas, May 2018.
  4. U-Net: Convolutional Networks for Biomedical Image Segmentation. In Nassir Navab, Joachim Hornegger, William M. Wells, and Alejandro F. Frangi, editors, Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, Lecture Notes in Computer Science, pages 234–241, Cham, 2015. Springer International Publishing. ISBN 978-3-319-24574-4. doi:10.1007/978-3-319-24574-4_28.
  5. Learned Primal-Dual Reconstruction. IEEE Transactions on Medical Imaging, 37(6):1322–1332, June 2018. ISSN 0278-0062, 1558-254X. doi:10.1109/TMI.2018.2799231.
  6. MoDL: Model Based Deep Learning Architecture for Inverse Problems. IEEE transactions on medical imaging, 38(2):394–405, February 2019. ISSN 0278-0062. doi:10.1109/TMI.2018.2865356.
  7. Learning a variational network for reconstruction of accelerated MRI data. Magnetic Resonance in Medicine, 79(6):3055–3071, 2018. ISSN 1522-2594. doi:10.1002/mrm.26977.
  8. Recurrent inference machines for reconstructing heterogeneous MRI data. Medical Image Analysis, 53:64–78, April 2019. ISSN 1361-8415. doi:10.1016/j.media.2019.01.005.
  9. A unified model for reconstruction and R2* mapping of accelerated 7T data using the quantitative recurrent inference machine. NeuroImage, 264:119680, December 2022. ISSN 1053-8119. doi:10.1016/j.neuroimage.2022.119680.
  10. Rich Caruana. Multitask Learning. Machine Learning, 28(1):41–75, July 1997. ISSN 1573-0565. doi:10.1023/A:1007379606734.
  11. Brain Segmentation from k-Space with End-to-End Recurrent Attention Network. In Dinggang Shen, Tianming Liu, Terry M. Peters, Lawrence H. Staib, Caroline Essert, Sean Zhou, Pew-Thian Yap, and Ali Khan, editors, Medical Image Computing and Computer Assisted Intervention – MICCAI 2019, Lecture Notes in Computer Science, pages 275–283, Cham, 2019. Springer International Publishing. ISBN 978-3-030-32248-9. doi:10.1007/978-3-030-32248-9_31.
  12. MultiTask Learning for accelerated-MRI Reconstruction and Segmentation of Brain Lesions in Multiple Sclerosis. In Medical Imaging with Deep Learning, pages 991–1005. PMLR, January 2024.
  13. RECONSTRUCTION AND SEGMENTATION OF PARALLEL MR DATA USING IMAGE DOMAIN DEEP-SLR. Proceedings. IEEE International Symposium on Biomedical Imaging, 2021:10.1109/isbi48211.2021.9434056, April 2021. ISSN 1945-7928. doi:10.1109/isbi48211.2021.9434056.
  14. Joint CS-MRI Reconstruction and Segmentation with a Unified Deep Network. In Albert C. S. Chung, James C. Gee, Paul A. Yushkevich, and Siqi Bao, editors, Information Processing in Medical Imaging, Lecture Notes in Computer Science, pages 492–504, Cham, 2019. Springer International Publishing. ISBN 978-3-030-20351-1. doi:10.1007/978-3-030-20351-1_38.
  15. Deep, deep learning with BART. Magnetic Resonance in Medicine, 89(2):678–693, 2023. ISSN 1522-2594. doi:10.1002/mrm.29485.
  16. DIRECT: Deep Image REConstruction Toolkit. Journal of Open Source Software, 7(73):4278, May 2022a. ISSN 2475-9066. doi:10.21105/joss.04278.
  17. NiftyNet: A deep-learning platform for medical imaging. Computer Methods and Programs in Biomedicine, 158:113–122, May 2018. ISSN 0169-2607. doi:10.1016/j.cmpb.2018.01.025.
  18. PyMIC: A deep learning toolkit for annotation-efficient medical image segmentation. Computer Methods and Programs in Biomedicine, 231:107398, April 2023. ISSN 0169-2607. doi:10.1016/j.cmpb.2023.107398.
  19. ClinicaDL: An open-source deep learning software for reproducible neuroimaging processing. Computer Methods and Programs in Biomedicine, 220:106818, June 2022. ISSN 0169-2607. doi:10.1016/j.cmpb.2022.106818.
  20. MRtrix3: A fast, flexible and open software framework for medical image processing and visualisation. NeuroImage, 202:116137, November 2019. ISSN 1053-8119. doi:10.1016/j.neuroimage.2019.116137.
  21. The ANTsX ecosystem for quantitative biological and medical imaging. Scientific Reports, 11(1):9068, April 2021. ISSN 2045-2322. doi:10.1038/s41598-021-87564-6.
  22. TorchIO: A Python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning. Computer Methods and Programs in Biomedicine, 208:106236, September 2021. ISSN 0169-2607. doi:10.1016/j.cmpb.2021.106236.
  23. MONAI: An open-source framework for deep learning in healthcare, November 2022.
  24. NeMo: A toolkit for building AI applications using Neural Modules, September 2019.
  25. Task adapted reconstruction for inverse problems. Inverse Problems, 38(7):075006, May 2022. ISSN 0266-5611. doi:10.1088/1361-6420/ac28ec.
  26. Results of the 2020 fastMRI Challenge for Machine Learning MR Image Reconstruction. IEEE transactions on medical imaging, 40(9):2306–2317, September 2021. ISSN 0278-0062. doi:10.1109/TMI.2021.3075856.
  27. Multi-Coil MRI Reconstruction Challenge-Assessing Brain MRI Reconstruction Models and Their Generalizability to Varying Coil Configurations. Frontiers in Neuroscience, 16:919186, 2022. ISSN 1662-4548. doi:10.3389/fnins.2022.919186.
  28. Gadgetron: An open source framework for medical image reconstruction. Magnetic Resonance in Medicine, 69(6):1768–1776, 2013. ISSN 1522-2594. doi:10.1002/mrm.24389.
  29. Coil compression for accelerated imaging with Cartesian sampling. Magnetic Resonance in Medicine, 69(2):571–582, 2013. ISSN 1522-2594. doi:10.1002/mrm.24267.
  30. SENSE: Sensitivity encoding for fast MRI. Magnetic Resonance in Medicine, 42(5):952–962, 1999. ISSN 1522-2594. doi:10.1002/(SICI)1522-2594(199911)42:5<952::AID-MRM16>3.0.CO;2-S.
  31. ESPIRiT — An Eigenvalue Approach to Autocalibrating Parallel MRI: Where SENSE meets GRAPPA. Magnetic resonance in medicine : official journal of the Society of Magnetic Resonance in Medicine / Society of Magnetic Resonance in Medicine, 71(3):990–1001, March 2014. ISSN 0740-3194. doi:10.1002/mrm.24751.
  32. Joint Deep Model-based MR Image and Coil Sensitivity Reconstruction Network (Joint-ICNet) for Fast MRI. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5266–5275, Nashville, TN, USA, June 2021. IEEE. ISBN 978-1-66544-509-2. doi:10.1109/CVPR46437.2021.00523.
  33. End-to-End Variational Networks for Accelerated MRI Reconstruction. In Anne L. Martel, Purang Abolmaesumi, Danail Stoyanov, Diana Mateus, Maria A. Zuluaga, S. Kevin Zhou, Daniel Racoceanu, and Leo Joskowicz, editors, Medical Image Computing and Computer Assisted Intervention – MICCAI 2020, Lecture Notes in Computer Science, pages 64–73, Cham, 2020. Springer International Publishing. ISBN 978-3-030-59713-9. doi:10.1007/978-3-030-59713-9_7.
  34. fastMRI: An Open Dataset and Benchmarks for Accelerated MRI, December 2019.
  35. Creation of Fully Sampled MR Data Repository for Compressed Sensing of the Knee.
  36. The Amsterdam Ultra-high field adult lifespan database (AHEAD): A freely available multimodal 7 Tesla submillimeter magnetic resonance imaging database. NeuroImage, 221:117200, November 2020. ISSN 1053-8119. doi:10.1016/j.neuroimage.2020.117200.
  37. MP2RAGEME: T1, T2*, and QSM mapping in one sequence at 7 tesla. Human Brain Mapping, 40(6):1786–1798, 2019. ISSN 1097-0193. doi:10.1002/hbm.24490.
  38. The Brain Tumor Segmentation (BraTS) Challenge 2023: Focus on Pediatrics (CBTN-CONNECT-DIPGR-ASNR-MICCAI BraTS-PEDs). ArXiv, page arXiv:2305.17033v5, February 2024. ISSN 2331-8422.
  39. ISLES 2022: A multi-center magnetic resonance imaging stroke lesion segmentation dataset. Scientific Data, 9(1):762, December 2022. ISSN 2052-4463. doi:10.1038/s41597-022-01875-5.
  40. SKM-TEA: A Dataset for Accelerated MRI Reconstruction with Dense Image Labels for Quantitative Clinical Evaluation, March 2022.
  41. Marco Cuturi. Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances, June 2013.
  42. Assessment of data consistency through cascades of independently recurrent inference machines for fast and robust accelerated MRI reconstruction. Physics in Medicine & Biology, 67(12):124001, June 2022. ISSN 0031-9155. doi:10.1088/1361-6560/ac6cc2.
  43. Convolutional Recurrent Neural Networks for Dynamic MR Image Reconstruction. IEEE Transactions on Medical Imaging, 38(1):280–290, January 2019. ISSN 0278-0062, 1558-254X. doi:10.1109/TMI.2018.2863670.
  44. A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstruction. IEEE Transactions on Medical Imaging, 37(2):491–503, February 2018. ISSN 0278-0062, 1558-254X. doi:10.1109/TMI.2017.2760978.
  45. KIKI-net: Cross-domain convolutional neural networks for reconstructing undersampled magnetic resonance images. Magnetic Resonance in Medicine, 80(5):2188–2201, 2018. ISSN 1522-2594. doi:10.1002/mrm.27201.
  46. Recurrent Variational Network: A Deep Learning Inverse Problem Solver applied to the task of Accelerated MRI Reconstruction. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 722–731, New Orleans, LA, USA, June 2022b. IEEE. ISBN 978-1-66546-946-3. doi:10.1109/CVPR52688.2022.00081.
  47. VS-Net: Variable splitting network for accelerated parallel MRI reconstruction, July 2019.
  48. Benchmarking MRI Reconstruction Neural Networks on Large Public Datasets. Applied Sciences, 10(5):1816, March 2020. ISSN 2076-3417. doi:10.3390/app10051816.
  49. Image Quality Assessment: From Error Visibility to Structural Similarity. IEEE Transactions on Image Processing, 13(4):600–612, April 2004. ISSN 1057-7149. doi:10.1109/TIP.2003.819861.
  50. ISMRM Raw data format: A proposed standard for MRI raw datasets. Magnetic Resonance in Medicine, 77(1):411–421, 2017. ISSN 1522-2594. doi:10.1002/mrm.26089.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets