2000 character limit reached
NeRF Solves Undersampled MRI Reconstruction (2402.13226v2)
Published 20 Feb 2024 in eess.IV, cs.AI, cs.CE, and eess.SP
Abstract: This article presents a novel undersampled magnetic resonance imaging (MRI) technique that leverages the concept of Neural Radiance Field (NeRF). With radial undersampling, the corresponding imaging problem can be reformulated into an image modeling task from sparse-view rendered data; therefore, a high dimensional MR image is obtainable from undersampled k-space data by taking advantage of implicit neural representation. A multi-layer perceptron, which is designed to output an image intensity from a spatial coordinate, learns the MR physics-driven rendering relation between given measurement data and desired image. Effective undersampling strategies for high-quality neural representation are investigated. The proposed method serves two benefits: (i) The learning is based fully on single undersampled k-space data, not a bunch of measured data and target image sets. It can be used potentially for diagnostic MR imaging, such as fetal MRI, where data acquisition is relatively rare or limited against diversity of clinical images while undersampled reconstruction is highly demanded. (ii) A reconstructed MR image is a scan-specific representation highly adaptive to the given k-space measurement. Numerous experiments validate the feasibility and capability of the proposed approach.
- E. M. Haacke, “Magnetic resonance imaging: physical principles and sequence design,” 1999.
- D. K. Sodickson and W. J. Manning, “Simultaneous acquisition of spatial harmonics (smash): fast imaging with radiofrequency coil arrays,” Magnetic resonance in medicine, vol. 38, no. 4, pp. 591–603, 1997.
- K. P. Pruessmann, M. Weiger, M. B. Scheidegger, and P. Boesiger, “Sense: sensitivity encoding for fast mri,” Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine, vol. 42, no. 5, pp. 952–962, 1999.
- C. M. Hyun and J. K. Seo, “Deep learning for ill posed inverse problems in medical imaging,” in Deep Learning and Medical Applications. Springer, 2023, pp. 319–339.
- H. Nyquist, “Certain topics in telegraph transmission theory,” Transactions of the American Institute of Electrical Engineers, vol. 47, no. 2, pp. 617–644, 1928.
- Y. Chen, C.-B. Schonlieb, P. Lio, T. Leiner, P. L. Dragotti, G. Wang, D. Rueckert, D. Firmin, and G. Yang, “Ai-based reconstruction for fast mri—a systematic review and meta-analysis,” Proceedings of the IEEE, vol. 110, no. 2, pp. 224–245, 2022.
- F. Knoll, T. Murrell, A. Sriram, N. Yakubova, J. Zbontar, M. Rabbat, A. Defazio, M. J. Muckley, D. K. Sodickson, C. L. Zitnick et al., “Advancing machine learning for mr image reconstruction with an open competition: Overview of the 2019 fastmri challenge,” Magnetic resonance in medicine, vol. 84, no. 6, pp. 3054–3070, 2020.
- M. J. Muckley, B. Riemenschneider, A. Radmanesh, S. Kim, G. Jeong, J. Ko, Y. Jun, H. Shin, D. Hwang, M. Mostapha et al., “Results of the 2020 fastmri challenge for machine learning mr image reconstruction,” IEEE transactions on medical imaging, vol. 40, no. 9, pp. 2306–2317, 2021.
- C. M. Hyun, S. H. Baek, M. Lee, S. M. Lee, and J. K. Seo, “Deep learning-based solvability of underdetermined inverse problems in medical imaging,” Medical Image Analysis, vol. 69, p. 101967, 2021.
- C. M. Hyun, H. P. Kim, S. M. Lee, S. Lee, and J. K. Seo, “Deep learning for undersampled mri reconstruction,” Physics in Medicine & Biology, vol. 63, no. 13, p. 135007, 2018.
- K. Deng, A. Liu, J.-Y. Zhu, and D. Ramanan, “Depth-supervised nerf: Fewer views and faster training for free,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 12 882–12 891.
- B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng, “Nerf: Representing scenes as neural radiance fields for view synthesis,” Communications of the ACM, vol. 65, no. 1, pp. 99–106, 2021.
- R. Martin-Brualla, N. Radwan, M. S. Sajjadi, J. T. Barron, A. Dosovitskiy, and D. Duckworth, “Nerf in the wild: Neural radiance fields for unconstrained photo collections,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 7210–7219.
- W. Yang, G. Chen, C. Chen, Z. Chen, and K.-Y. K. Wong, “Ps-nerf: Neural inverse rendering for multi-view photometric stereo,” in European Conference on Computer Vision. Springer, 2022, pp. 266–284.
- J. Zbontar, F. Knoll, A. Sriram, T. Murrell, Z. Huang, M. J. Muckley, A. Defazio, R. Stern, P. Johnson, M. Bruno et al., “fastmri: An open dataset and benchmarks for accelerated mri,” arXiv preprint arXiv:1811.08839, 2018.
- S. N. Saleem, “Fetal mri: An approach to practice: A review,” Journal of advanced research, vol. 5, no. 5, pp. 507–523, 2014.
- L. Manganaro, S. Capuani, M. Gennarini, V. Miceli, R. Ninkova, I. Balba, N. Galea, A. Cupertino, A. Maiuro, G. Ercolani et al., “Fetal mri: what’s new? a short review,” European Radiology Experimental, vol. 7, no. 1, p. 41, 2023.
- L. Feng, “Golden-angle radial mri: basics, advances, and applications,” Journal of Magnetic Resonance Imaging, vol. 56, no. 1, pp. 45–62, 2022.
- C. M. Hyun, K. C. Kim, H. C. Cho, J. K. Choi, and J. K. Seo, “Framelet pooling aided deep learning network: the method to process high dimensional medical data,” Machine Learning: Science and Technology, vol. 1, no. 1, p. 015009, 2020.
- C. M. Hyun, T. Bayaraa, S. M. Lee, H. Jung, and J. K. Seo, “Deep learning for dental cone-beam computed tomography,” Deep Learning and Medical Applications, pp. 101–175, 2023.
- M. Tancik, P. Srinivasan, B. Mildenhall, S. Fridovich-Keil, N. Raghavan, U. Singhal, R. Ramamoorthi, J. Barron, and R. Ng, “Fourier features let networks learn high frequency functions in low dimensional domains,” Advances in Neural Information Processing Systems, vol. 33, pp. 7537–7547, 2020.
- G. F. Montufar, R. Pascanu, K. Cho, and Y. Bengio, “On the number of linear regions of deep neural networks,” Advances in neural information processing systems, vol. 27, 2014.
- J. Bruna and S. Mallat, “Invariant scattering convolution networks,” IEEE transactions on pattern analysis and machine intelligence, vol. 35, no. 8, pp. 1872–1886, 2013.
- M. Vassholz, B. Koberstein-Schwarz, A. Ruhlandt, M. Krenkel, and T. Salditt, “New x-ray tomography method based on the 3d radon transform compatible with anisotropic sources,” Physical Review Letters, vol. 116, no. 8, p. 088101, 2016.
- S. Zhang, K. T. Block, and J. Frahm, “Magnetic resonance imaging in real time: advances using radial flash,” Journal of Magnetic Resonance Imaging, vol. 31, no. 1, pp. 101–109, 2010.
- J. Chen, Y. Lu, Q. Yu, X. Luo, E. Adeli, Y. Wang, L. Lu, A. L. Yuille, and Y. Zhou, “Transunet: Transformers make strong encoders for medical image segmentation,” arXiv preprint arXiv:2102.04306, 2021.
- W. Huang, H. B. Li, J. Pan, G. Cruz, D. Rueckert, and K. Hammernik, “Neural implicit k-space for binning-free non-cartesian cardiac mr imaging,” in International Conference on Information Processing in Medical Imaging. Springer, 2023, pp. 548–560.
- F. Ong and M. Lustig, “Sigpy: a python package for high performance iterative reconstruction,” in Proceedings of the ISMRM 27th Annual Meeting, Montreal, Quebec, Canada, vol. 4819, no. 5, 2019.
- S. Winkelmann, T. Schaeffter, T. Koehler, H. Eggers, and O. Doessel, “An optimal radial profile order based on the golden ratio for time-resolved mri,” IEEE transactions on medical imaging, vol. 26, no. 1, pp. 68–76, 2006.
- L. Shen, J. Pauly, and L. Xing, “Nerp: implicit neural representation learning with prior embedding for sparsely sampled image reconstruction,” IEEE Transactions on Neural Networks and Learning Systems, 2022.
- Tae Jun Jang (6 papers)
- Chang Min Hyun (11 papers)