Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
134 tokens/sec
GPT-4o
9 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

PINQI: An End-to-End Physics-Informed Approach to Learned Quantitative MRI Reconstruction (2306.11023v2)

Published 19 Jun 2023 in eess.IV, cs.CV, cs.LG, and physics.med-ph

Abstract: Quantitative Magnetic Resonance Imaging (qMRI) enables the reproducible measurement of biophysical parameters in tissue. The challenge lies in solving a nonlinear, ill-posed inverse problem to obtain the desired tissue parameter maps from acquired raw data. While various learned and non-learned approaches have been proposed, the existing learned methods fail to fully exploit the prior knowledge about the underlying MR physics, i.e. the signal model and the acquisition model. In this paper, we propose PINQI, a novel qMRI reconstruction method that integrates the knowledge about the signal, acquisition model, and learned regularization into a single end-to-end trainable neural network. Our approach is based on unrolled alternating optimization, utilizing differentiable optimization blocks to solve inner linear and non-linear optimization tasks, as well as convolutional layers for regularization of the intermediate qualitative images and parameter maps. This design enables PINQI to leverage the advantages of both the signal model and learned regularization. We evaluate the performance of our proposed network by comparing it with recently published approaches in the context of highly undersampled $T_1$-mapping, using both a simulated brain dataset, as well as real scanner data acquired from a physical phantom and in-vivo data from healthy volunteers. The results demonstrate the superiority of our proposed solution over existing methods and highlight the effectiveness of our method in real-world scenarios.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (61)
  1. K. P. Pruessmann, M. Weiger, M. B. Scheidegger, and P. Boesiger, “SENSE: Sensitivity encoding for fast MRI,” Magnetic Resonance in Medicine, vol. 42, no. 5, pp. 952–962, 1999. 10.1002/(SICI)1522-2594(199911)42:5¡952::AID-MRM16¿3.0.CO;2-S
  2. M. Lustig, D. L. Donoho, J. M. Santos, and J. M. Pauly, “Compressed Sensing MRI,” IEEE Signal Processing Magazine, vol. 25, no. 2, pp. 72–82, 2008. 10.1109/MSP.2007.914728
  3. M. Doneva, P. Börnert, H. Eggers, C. Stehning, J. Sénégas, and A. Mertins, “Compressed sensing reconstruction for magnetic resonance parameter mapping,” Magnetic Resonance in Medicine, vol. 64, no. 4, pp. 1114–1120, 2010. 10.1002/mrm.22483
  4. X. Wang, V. Roeloffs, J. Klosowski, Z. Tan, D. Voit, M. Uecker, and J. Frahm, “Model-based T1 mapping with sparsity constraints using single-shot inversion-recovery radial FLASH,” Magnetic Resonance in Medicine, vol. 79, no. 2, pp. 730–740, 2018. 10.1002/mrm.26726
  5. B. Zhao, F. Lam, and Z. P. Liang, “Model-based MR parameter mapping with sparsity constraints: Parameter estimation and performance bounds,” IEEE Transactions on Medical Imaging, vol. 33, no. 9, pp. 1832–1844, 2014. 10.1109/TMI.2014.2322815
  6. K. T. Block, M. Uecker, and J. Frahm, “Undersampled radial MRI with multiple coils. Iterative image reconstruction using a total variation constraint,” Magnetic Resonance in Medicine, vol. 57, no. 6, pp. 1086–1098, 2007. 10.1002/mrm.21236
  7. J. Tran-Gia, D. Stäb, T. Wech, D. Hahn, and H. Köstler, “Model-based Acceleration of Parameter mapping (MAP) for saturation prepared radially acquired data,” Magnetic Resonance in Medicine, vol. 70, no. 6, pp. 1524–1534, 2013. 10.1002/mrm.24600
  8. K. M. Becker, J. Schulz-Menger, T. Schaeffter, and C. Kolbitsch, “Simultaneous high-resolution cardiac T1 mapping and cine imaging using model-based iterative image reconstruction,” Magnetic Resonance in Medicine, vol. 81, no. 2, pp. 1080–1091, 2019. 10.1002/mrm.27474
  9. A. Sriram, J. Zbontar, T. Murrell, A. Defazio, C. L. Zitnick, N. Yakubova, F. Knoll, and P. Johnson, “End-to-End Variational Networks for Accelerated MRI Reconstruction,” Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. Lecture Notes in Computer Science, vol. 12262 LNCS, pp. 64–73, 2020. 10.1007/978-3-030-59713-9_7
  10. H. K. Aggarwal, M. P. Mani, and M. Jacob, “MoDL: Model-Based Deep Learning Architecture for Inverse Problems,” IEEE Transactions on Medical Imaging, vol. 38, no. 2, pp. 394–405, 2019. 10.1109/TMI.2018.2865356
  11. Y. Jun, H. Shin, T. Eo, T. Kim, and D. Hwang, “Deep model-based magnetic resonance parameter mapping network (DOPAMINE) for fast T1 mapping using variable flip angle method,” Medical Image Analysis, vol. 70, p. 102017, 2021. 10.1016/j.media.2021.102017
  12. H. Jeelani, Y. Yang, R. Zhou, C. M. Kramer, M. Salerno, and D. S. Weller, “A Myocardial T1-Mapping Framework with Recurrent and U-Net Convolutional Neural Networks,” Proceedings - International Symposium on Biomedical Imaging, vol. 2020-April, pp. 1941–1944, 2020. 10.1109/ISBI45749.2020.9098459
  13. X. Xu, W. Gan, S. V. Kothapalli, D. A. Yablonskiy, and U. S. Kamilov, “CoRRECT: A Deep Unfolding Framework for Motion-Corrected Quantitative R2* Mapping,” 2022. 10.48550/arXiv.2210.06330
  14. F. Liu, L. Feng, and R. Kijowski, “MANTIS: Model-Augmented Neural neTwork with Incoherent k-space Sampling for efficient MR parameter mapping,” Magnetic Resonance in Medicine, vol. 82, no. 1, pp. 174–188, 2019. 10.1002/mrm.27707
  15. R. Guo, H. El-Rewaidy, S. Assana, X. Cai, A. Amyar, K. Chow, X. Bi, T. Yankama, J. Cirillo, P. Pierce, B. Goddu, L. Ngo, and R. Nezafat, “Accelerated cardiac T1 mapping in four heartbeats with inline MyoMapNet: a deep learning-based T1 estimation approach,” Journal of Cardiovascular Magnetic Resonance, vol. 24, no. 1, pp. 1–15, 2022. 10.1186/s12968-021-00834-0
  16. O. Cohen, B. Zhu, and M. S. Rosen, “MR fingerprinting Deep RecOnstruction NEtwork (DRONE),” Magnetic Resonance in Medicine, vol. 80, no. 3, pp. 885–894, 2018. 10.1002/mrm.27198
  17. J. Schlemper, J. Caballero, J. V. Hajnal, A. N. Price, and D. Rueckert, “A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstruction,” IEEE Transactions on Medical Imaging, vol. 37, no. 2, pp. 491–503, 2018. 10.1007/978-3-319-59050-951
  18. C. Qin, J. Schlemper, J. Caballero, A. N. Price, J. V. Hajnal, and D. Rueckert, “Convolutional recurrent neural networks for dynamic MR image reconstruction,” IEEE Transactions on Medical Imaging, vol. 38, no. 1, pp. 280–290, 2019. 10.1109/TMI.2018.2863670
  19. Y. Yang, J. Sun, H. Li, and Z. Xu, “Deep ADMM-Net for compressive sensing MRI,” Advances in Neural Information Processing Systems, vol. 29, 2016. 10.5555/3157096.3157098
  20. K. Chow, J. A. Flewitt, J. D. Green, J. J. Pagano, M. G. Friedrich, and R. B. Thompson, “Saturation recovery single-shot acquisition (SASHA) for myocardial T 1 mapping,” Magnetic Resonance in Medicine, vol. 71, no. 6, pp. 2082–2095, 2014. 10.1002/mrm.24878
  21. J. Z. Bojorquez, S. Bricq, C. Acquitter, F. Brunotte, P. M. Walker, and A. Lalande, “What are normal relaxation times of tissues at 3 T?” Magnetic Resonance Imaging, vol. 35, pp. 69–80, 2017. 10.1016/j.mri.2016.08.021
  22. D. C. Liu and J. Nocedal, “On the limited memory BFGS method for large scale optimization,” Mathematical Programming, vol. 45, no. 1-3, pp. 503–528, 8 1989. 10.1007/BF01589116
  23. D. Ma, V. Gulani, N. Seiberlich, K. Liu, J. L. Sunshine, J. L. Duerk, and M. A. Griswold, “Magnetic resonance fingerprinting,” Nature, vol. 495, no. 7440, pp. 187–192, 3 2013. 10.1038/nature11971
  24. H. Li, M. Yang, J. H. Kim, C. Zhang, R. Liu, P. Huang, D. Liang, X. Zhang, X. Li, and L. Ying, “SuperMAP: Deep ultrafast MR relaxometry with joint spatiotemporal undersampling,” Magnetic Resonance in Medicine, vol. 89, no. 1, pp. 64–76, 2023. 10.1002/mrm.29411
  25. O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9351, pp. 234–241, 2015. 10.1007/978-3-319-24574-4_28
  26. K. Hammernik, T. Kustner, B. Yaman, Z. Huang, D. Rueckert, F. Knoll, and M. Akcakaya, “Physics-Driven Deep Learning for Computational Magnetic Resonance Imaging: Combining physics and machine learning for improved medical imaging,” IEEE Signal Processing Magazine, vol. 40, no. 1, pp. 98–114, 2023. 10.1109/msp.2022.3215288
  27. D. Chen, M. E. Davies, and M. Golbabaee, “Deep Unrolling for Magnetic Resonance Fingerprinting,” Proceedings - International Symposium on Biomedical Imaging, no. 2, 2022. 10.1109/ISBI52829.2022.9761475
  28. A. Kofler, M. Haltmeier, T. Schaeffter, and C. Kolbitsch, “An end-to-end-trainable iterative network architecture for accelerated radial multi-coil 2d cine mr image reconstruction,” Medical Physics, vol. 48, no. 5, pp. 2412–2425, 2021.
  29. B. Amos and J. Z. Kolter, “OptNet: Differentiable optimization as a layer in neural networks,” 34th International Conference on Machine Learning, ICML 2017, vol. 1, pp. 179–191, 2017. 10.48550/arXiv.1703.0044
  30. A. Agrawal, B. Amos, S. Barratt, S. Boyd, S. Diamond, and J. Zico Kolter, “Differentiable convex optimization layers,” Advances in Neural Information Processing Systems, vol. 32, no. NeurIPS, 2019. 10.48550/arXiv.1910.12430
  31. Z. Lv, F. Dellaert, J. M. Rehg, and A. Geiger, “Taking a deeper look at the inverse compositional algorithm,” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2019-June, pp. 4576–4585, 2019. 10.1109/CVPR.2019.00471
  32. K. Hammernik, T. Küstner, and D. Rueckert, “Machine Learning for MRI Reconstruction,” in Magnetic Resonance Image Reconstruction, C. Prieto, M. I. Doneva, and M. Akcakaya, Eds.   Elsevier, 2022, ch. 11, pp. 281–317.
  33. S. Gould, B. Fernando, A. Cherian, P. Anderson, R. S. Cruz, and E. Guo, “On differentiating parameterized argmin and argmax problems with application to bi-level optimization,” 2016.
  34. O. de Oliveira, “The implicit and the inverse function theorems: Easy proofs,” Real Analysis Exchange, vol. 39, no. 1, pp. 207–218, 2013. 10.14321/realanalexch.39.1.0207
  35. F. Pedregosa, “Hyperparameter optimization with approximate gradient,” 33rd International Conference on Machine Learning, ICML 2016, vol. 2, pp. 1150–1159, 2016. 10.48550/arXiv.1602.02355
  36. M. V. Afonso, J. M. Bioucas-Dias, and M. A. Figueiredo, “Fast image recovery using variable splitting and constrained optimization,” IEEE Transactions on Image Processing, vol. 19, no. 9, pp. 2345–2356, 2010. 10.1109/TIP.2010.2047910
  37. K. Kreutz-Delgado, “The complex gradient operator and the CR-calculus,” arXiv preprint, 2009. 10.48550/arXiv.0906.4835
  38. B. Aubert-Broche, M. Griffin, G. B. Pike, A. C. Evans, and D. L. Collins, “Twenty new digital brain phantoms for creation of validation image data bases,” IEEE Transactions on Medical Imaging, vol. 25, no. 11, pp. 1410–1416, 2006. 10.1109/TMI.2006.883453
  39. M. J. Muckley, R. Stern, T. Murrell, and F. Knoll, “mrisensesim.py,” TorchKbNufft: A High-Level, Hardware-Agnostic Non-Uniform Fast Fourier Transform, ISMRM Workshop on Data Sampling & Image Reconstruction. Online github.com/mmuckley/torchkbnufft/tree/v0.3.4/torchkbnufft/mri.
  40. D. O. Walsh, A. F. Gmitro, and M. W. Marcellin, “Adaptive reconstruction of phased array MR imagery,” Magnetic Resonance in Medicine, vol. 43, no. 5, pp. 682–690, 2000. 10.1002/(SICI)1522-2594(200005)43:5¡682::AID-MRM10¿3.0.CO;2-G
  41. S. J. Inati, M. S. Hansen, and P. Kellman, “A Fast Optimal Method for Coil Sensitivity Estimation and Adaptive Coil Combination for Complex Images,” Proceedings of the 22nd Annual Meeting of ISMRM, 2014.
  42. M. Uecker, P. Lai, M. J. Murphy, P. Virtue, M. Elad, J. M. Pauly, S. S. Vasanawala, and M. Lustig, “ESPIRiT–an eigenvalue approach to autocalibrating parallel MRI: where SENSE meets GRAPPA,” Magnetic Resonance in Medicine, vol. 71, no. 3, pp. 990–1001, 2014. 10.1002/mrm.24751
  43. K. J. Layton, S. Kroboth, F. Jia, S. Littin, H. Yu, J. Leupold, J.-F. Nielsen, T. Stöcker, and M. Zaitsev, “Pulseq: A rapid and hardware-independent pulse sequence prototyping framework,” Magnetic Resonance in Medicine, vol. 77, no. 4, pp. 1544–1552, 2017. 10.1002/mrm.26235
  44. B. Zoph and Q. V. Le, “Searching for activation functions,” 6th International Conference on Learning Representations, ICLR 2018 - Workshop Track Proceedings, no. 1, pp. 1–12, 2018. 10.48550/arXiv.1710.05941
  45. Y. Wu and K. He, “Group Normalization,” International Journal of Computer Vision, vol. 128, no. 3, pp. 742–755, 2020. 10.1007/s11263-019-01198-w
  46. Z. Qiu, T. Yao, and T. Mei, “Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks,” Proceedings of the IEEE International Conference on Computer Vision, vol. 2017-Octob, pp. 5534–5542, 10 2017. 10.1109/ICCV.2017.590
  47. E. Perez, F. Strub, H. De Vries, V. Dumoulin, and A. Courville, “FiLM: Visual reasoning with a general conditioning layer,” 32nd AAAI Conference on Artificial Intelligence, AAAI 2018, pp. 3942–3951, 2018. 10.1609/aaai.v32i1.11671
  48. F. F. Zimmermann and A. Kofler, “NoSENSE: Learned unrolled cardiac MRI reconstruction without explicit sensitivity maps,” International Workshop on Statistical Atlases and Computational Models of the Heart (STACOM), 2023. 10.48550/arXiv.2309.15608
  49. T. Bachlechner, B. P. Majumder, H. Mao, G. Cottrell, and J. McAuley, “ReZero is All You Need: Fast Convergence at Large Depth,” 37th Conference on Uncertainty in Artificial Intelligence, UAI 2021, no. 1, pp. 1352–1361, 2021. 10.48550/arXiv./2003.04887
  50. B. Zhou and S. Kevin Zhou, “Dudornet: Learning a dual-domain recurrent network for fast MRI reconstruction with deep T1 prior,” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 4272–4281, 2020. 10.1109/CVPR42600.2020.00433
  51. L. Wang, C.-Y. Lee, Z. Tu, and S. Lazebnik, “Training deeper convolutional networks with deep supervision,” 2015.
  52. D. P. Kingma and J. L. Ba, “Adam: A method for stochastic optimization,” 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings, pp. 1–15, 2015. 10.48550/arXiv.412.6980
  53. T. He, Z. Zhang, H. Zhang, Z. Zhang, J. Xie, and M. Li, “Bag of tricks for image classification with convolutional neural networks,” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2019-June, pp. 558–567, 2019. 10.48550/arXiv.1812.01187
  54. I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” 7th International Conference on Learning Representations, ICLR 2019, 2019. 10.48550/arXiv.1711.05101
  55. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE transactions on image processing, vol. 13, no. 4, pp. 600–612, 2004.
  56. X. Zhang, Q. Duchemin, K. Liu*, C. Gultekin, S. Flassbeck, C. Fernandez-Granda, and J. Assländer, “Cramér–Rao bound-informed training of neural networks for quantitative MRI,” Magnetic Resonance in Medicine, vol. 88, no. 1, pp. 436–448, 2022. 10.1002/mrm.29206
  57. A. Kendall, Y. Gal, and R. Cipolla, “Multi-task learning using uncertainty to weigh losses for scene geometry and semantics,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018. 10.48550/arXiv.1705.07115 pp. 7482–7491.
  58. D. Gilton, G. Ongie, and R. Willett, “Model Adaptation for Inverse Problems in Imaging,” IEEE Transactions on Computational Imaging, vol. 7, no. 2, pp. 661–674, 2021. 10.1109/TCI.2021.3094714
  59. K. Hammernik, J. Schlemper, C. Qin, J. Duan, R. M. Summers, and D. Rueckert, “Systematic evaluation of iterative deep neural networks for fast parallel mri reconstruction with sensitivity-weighted coil combination,” Magnetic Resonance in Medicine, vol. 86, no. 4, pp. 1859–1872, 2021. 10.1002/mrm.28827
  60. F. F. Zimmermann, A. Kofler, C. Kolbitsch, and P. Schuenke, “Semi-Supervised Learning for Spatially Regularized Quantitative MRI Reconstruction - Application to Simultaneous T1, B0, B1 Mapping,” 2023, 1166, ISMRM Annual Meeting.
  61. Y. Jun, J. Cho, X. Wang, M. Gee, P. E. Grant, B. Bilgic, and B. Gagoski, “SSL-QALAS: Self-Supervised Learning for rapid multiparameter estimation in quantitative MRI using 3D-QALAS,” Magnetic Resonance in Medicine, vol. 90, no. 5, pp. 2019–2032, 2023. 10.1002/mrm.29786
Citations (4)

Summary

We haven't generated a summary for this paper yet.