Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
157 tokens/sec
GPT-4o
43 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Graph Image Prior for Unsupervised Dynamic Cardiac Cine MRI Reconstruction (2403.15770v3)

Published 23 Mar 2024 in eess.IV and cs.CV

Abstract: The inductive bias of the convolutional neural network (CNN) can be a strong prior for image restoration, which is known as the Deep Image Prior (DIP). Recently, DIP is utilized in unsupervised dynamic MRI reconstruction, which adopts a generative model from the latent space to the image space. However, existing methods usually use a pyramid-shaped CNN generator shared by all frames, embedding the temporal modeling within the latent space, which may hamper the model expression capability. In this work, we propose a novel scheme for dynamic MRI representation, named ``Graph Image Prior'' (GIP). GIP adopts a two-stage generative network in a new modeling methodology, which first employs independent CNNs to recover the image structure for each frame, and then exploits the spatio-temporal correlations within the feature space parameterized by a graph model. A graph convolutional network is utilized for feature fusion and dynamic image generation. In addition, we devise an ADMM algorithm to alternately optimize the images and the network parameters to improve the reconstruction performance. Experiments were conducted on cardiac cine MRI reconstruction, which demonstrate that GIP outperforms compressed sensing methods and other DIP-based unsupervised methods, significantly reducing the performance gap with state-of-the-art supervised algorithms. Moreover, GIP displays superior generalization ability when transferred to a different reconstruction setting, without the need for any additional data.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (50)
  1. J. Tsao, P. Boesiger, and K. P. Pruessmann, “k-t blast and k-t sense: dynamic mri with high frame rate exploiting spatiotemporal correlations,” Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine, vol. 50, no. 5, pp. 1031–1042, 2003.
  2. Z.-P. Liang, “Spatiotemporal imagingwith partially separable functions,” in 2007 4th IEEE international symposium on biomedical imaging: from nano to macro.   IEEE, 2007, pp. 988–991.
  3. L. Feng, M. B. Srichai, R. P. Lim, A. Harrison, W. King, G. Adluru, E. V. Dibella, D. K. Sodickson, R. Otazo, and D. Kim, “Highly accelerated real-time cardiac cine mri using k–t sparse-sense,” Magnetic resonance in medicine, vol. 70, no. 1, pp. 64–74, 2013.
  4. H. Jung, K. Sung, K. S. Nayak, E. Y. Kim, and J. C. Ye, “k-t focuss: a general compressed sensing framework for high resolution dynamic mri,” Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine, vol. 61, no. 1, pp. 103–116, 2009.
  5. B. Zhao, J. P. Haldar, A. G. Christodoulou, and Z.-P. Liang, “Image reconstruction from highly undersampled (k, t)-space data with joint partial separability and sparsity constraints,” IEEE transactions on medical imaging, vol. 31, no. 9, pp. 1809–1820, 2012.
  6. S. G. Lingala, Y. Hu, E. DiBella, and M. Jacob, “Accelerated dynamic mri exploiting sparsity and low-rank structure: kt slr,” IEEE transactions on medical imaging, vol. 30, no. 5, pp. 1042–1054, 2011.
  7. R. Otazo, E. Candes, and D. K. Sodickson, “Low-rank plus sparse matrix decomposition for accelerated dynamic mri with separation of background and dynamic components,” Magnetic resonance in medicine, vol. 73, no. 3, pp. 1125–1136, 2015.
  8. S. Poddar and M. Jacob, “Dynamic mri using smoothness regularization on manifolds (storm),” IEEE transactions on medical imaging, vol. 35, no. 4, pp. 1106–1115, 2015.
  9. U. Nakarmi, Y. Wang, J. Lyu, D. Liang, and L. Ying, “A kernel-based low-rank (klr) model for low-dimensional manifold recovery in highly accelerated dynamic mri,” IEEE transactions on medical imaging, vol. 36, no. 11, pp. 2297–2307, 2017.
  10. S. Wang, Z. Ke, H. Cheng, S. Jia, L. Ying, H. Zheng, and D. Liang, “Dimension: dynamic mr imaging with both k-space and spatial prior knowledge obtained via multi-supervised network training,” NMR in Biomedicine, vol. 35, no. 4, p. e4131, 2022.
  11. Q. Huang, Y. Xian, D. Yang, H. Qu, J. Yi, P. Wu, and D. N. Metaxas, “Dynamic mri reconstruction with end-to-end motion-guided network,” Medical Image Analysis, vol. 68, p. 101901, 2021.
  12. J. Sun, H. Li, Z. Xu et al., “Deep admm-net for compressive sensing mri,” Advances in neural information processing systems, vol. 29, 2016.
  13. H. K. Aggarwal, M. P. Mani, and M. Jacob, “Modl: Model-based deep learning architecture for inverse problems,” IEEE transactions on medical imaging, vol. 38, no. 2, pp. 394–405, 2018.
  14. K. Hammernik, T. Klatzer, E. Kobler, M. P. Recht, D. K. Sodickson, T. Pock, and F. Knoll, “Learning a variational network for reconstruction of accelerated mri data,” Magnetic resonance in medicine, vol. 79, no. 6, pp. 3055–3071, 2018.
  15. J. Schlemper, J. Caballero, J. V. Hajnal, A. N. Price, and D. Rueckert, “A deep cascade of convolutional neural networks for dynamic mr image reconstruction,” IEEE transactions on Medical Imaging, vol. 37, no. 2, pp. 491–503, 2017.
  16. C. Qin, J. Schlemper, J. Caballero, A. N. Price, J. V. Hajnal, and D. Rueckert, “Convolutional recurrent neural networks for dynamic mr image reconstruction,” IEEE transactions on medical imaging, vol. 38, no. 1, pp. 280–290, 2018.
  17. C. Qin, J. Duan, K. Hammernik, J. Schlemper, T. Küstner, R. Botnar, C. Prieto, A. N. Price, J. V. Hajnal, and D. Rueckert, “Complementary time-frequency domain networks for dynamic parallel mr image reconstruction,” Magnetic Resonance in Medicine, vol. 86, no. 6, pp. 3274–3291, 2021.
  18. J. Zhang and B. Ghanem, “Ista-net: Interpretable optimization-inspired deep network for image compressive sensing,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 1828–1837.
  19. Z. Ke, W. Huang, Z.-X. Cui, J. Cheng, S. Jia, H. Wang, X. Liu, H. Zheng, L. Ying, Y. Zhu et al., “Learned low-rank priors in dynamic mr imaging,” IEEE Transactions on Medical Imaging, vol. 40, no. 12, pp. 3698–3710, 2021.
  20. W. Huang, Z. Ke, Z.-X. Cui, J. Cheng, Z. Qiu, S. Jia, L. Ying, Y. Zhu, and D. Liang, “Deep low-rank plus sparse network for dynamic mr imaging,” Medical Image Analysis, vol. 73, p. 102190, 2021.
  21. D. Ulyanov, A. Vedaldi, and V. Lempitsky, “Deep image prior,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 9446–9454.
  22. M. Z. Darestani and R. Heckel, “Accelerated mri with un-trained neural networks,” IEEE Transactions on Computational Imaging, vol. 7, pp. 724–733, 2021.
  23. K. Gong, C. Catana, J. Qi, and Q. Li, “Pet image reconstruction using deep image prior,” IEEE transactions on medical imaging, vol. 38, no. 7, pp. 1655–1665, 2018.
  24. J. Yoo, K. H. Jin, H. Gupta, J. Yerly, M. Stuber, and M. Unser, “Time-dependent deep image prior for dynamic mri,” IEEE Transactions on Medical Imaging, vol. 40, no. 12, pp. 3337–3348, 2021.
  25. Q. Zou, A. H. Ahmed, P. Nagpal, S. Kruger, and M. Jacob, “Dynamic imaging using a deep generative storm (gen-storm) model,” IEEE transactions on medical imaging, vol. 40, no. 11, pp. 3102–3112, 2021.
  26. A. H. Ahmed, Q. Zou, P. Nagpal, and M. Jacob, “Dynamic imaging using deep bi-linear unsupervised representation (deblur),” IEEE transactions on medical imaging, vol. 41, no. 10, pp. 2693–2703, 2022.
  27. Y. Q. Mohsin, S. Poddar, and M. Jacob, “Free-breathing & ungated cardiac mri using iterative storm (i-storm),” IEEE transactions on medical imaging, vol. 38, no. 10, pp. 2303–2313, 2019.
  28. S. Poddar, Y. Q. Mohsin, D. Ansah, B. Thattaliyath, R. Ashwath, and M. Jacob, “Manifold recovery using kernel low-rank regularization: Application to dynamic imaging,” IEEE transactions on computational imaging, vol. 5, no. 3, pp. 478–491, 2019.
  29. U. Nakarmi, K. Slavakis, and L. Ying, “Mls: Joint manifold-learning and sparsity-aware framework for highly accelerated dynamic magnetic resonance imaging,” in 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).   IEEE, 2018, pp. 1213–1216.
  30. Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang, and S. Y. Philip, “A comprehensive survey on graph neural networks,” IEEE transactions on neural networks and learning systems, vol. 32, no. 1, pp. 4–24, 2020.
  31. A. Sandryhaila and J. M. Moura, “Discrete signal processing on graphs,” IEEE transactions on signal processing, vol. 61, no. 7, pp. 1644–1656, 2013.
  32. M. Defferrard, X. Bresson, and P. Vandergheynst, “Convolutional neural networks on graphs with fast localized spectral filtering,” Advances in neural information processing systems, vol. 29, 2016.
  33. T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” arXiv preprint arXiv:1609.02907, 2016.
  34. G. Li, M. Muller, A. Thabet, and B. Ghanem, “Deepgcns: Can gcns go as deep as cnns?” in Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 9267–9276.
  35. K. Han, Y. Wang, J. Guo, Y. Tang, and E. Wu, “Vision gnn: An image is worth graph of nodes,” Advances in Neural Information Processing Systems, vol. 35, pp. 8291–8303, 2022.
  36. F. Cheng, Y. Liu, Y. Chen, and P.-T. Yap, “High-resolution 3d magnetic resonance fingerprinting with a graph convolutional network,” IEEE transactions on medical imaging, vol. 42, no. 3, pp. 674–683, 2022.
  37. H. Lu, H. Ye, L. L. Wald, and B. Zhao, “Accelerated mr fingerprinting with low-rank and generative subspace modeling,” arXiv preprint arXiv:2305.10651, 2023.
  38. C. Chen, Y. Liu, P. Schniter, M. Tong, K. Zareba, O. Simonetti, L. Potter, and R. Ahmad, “Ocmr (v1. 0)–open-access multi-coil k-space dataset for cardiovascular magnetic resonance imaging,” arXiv preprint arXiv:2008.03410, 2020.
  39. T. Zhang, J. M. Pauly, S. S. Vasanawala, and M. Lustig, “Coil compression for accelerated imaging with cartesian sampling,” Magnetic resonance in medicine, vol. 69, no. 2, pp. 571–582, 2013.
  40. C. Wang, J. Lyu, S. Wang, C. Qin, K. Guo, X. Zhang, X. Yu, Y. Li, F. Wang, J. Jin et al., “Cmrxrecon: an open cardiac mri dataset for the competition of accelerated image reconstruction,” arXiv preprint arXiv:2309.10836, 2023.
  41. M. Uecker, P. Lai, M. J. Murphy, P. Virtue, M. Elad, J. M. Pauly, S. S. Vasanawala, and M. Lustig, “Espirit—an eigenvalue approach to autocalibrating parallel mri: where sense meets grappa,” Magnetic resonance in medicine, vol. 71, no. 3, pp. 990–1001, 2014.
  42. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE transactions on image processing, vol. 13, no. 4, pp. 600–612, 2004.
  43. S. Winkelmann, T. Schaeffter, T. Koehler, H. Eggers, and O. Doessel, “An optimal radial profile order based on the golden ratio for time-resolved mri,” IEEE transactions on medical imaging, vol. 26, no. 1, pp. 68–76, 2006.
  44. M. J. Muckley, R. Stern, T. Murrell, and F. Knoll, “TorchKbNufft: A high-level, hardware-agnostic non-uniform fast Fourier transform,” in ISMRM Workshop on Data Sampling & Image Reconstruction, 2020, source code available at https://github.com/mmuckley/torchkbnufft.
  45. M. E. Arican, O. Kara, G. Bredell, and E. Konukoglu, “Isnas-dip: Image-specific neural architecture search for deep image prior,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 1960–1968.
  46. R. Heckel and P. Hand, “Deep decoder: Concise image representations from untrained non-convolutional networks,” arXiv preprint arXiv:1810.03982, 2018.
  47. Z. Cheng, M. Gadelha, S. Maji, and D. Sheldon, “A bayesian perspective on the deep image prior,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 5443–5451.
  48. G. Yang, S. Yu, H. Dong, G. Slabaugh, P. L. Dragotti, X. Ye, F. Liu, S. Arridge, J. Keegan, Y. Guo et al., “Dagan: Deep de-aliasing generative adversarial networks for fast compressed sensing mri reconstruction,” IEEE transactions on medical imaging, vol. 37, no. 6, pp. 1310–1321, 2017.
  49. R. G. Baraniuk, “Compressive sensing [lecture notes],” IEEE signal processing magazine, vol. 24, no. 4, pp. 118–121, 2007.
  50. M. Lustig, D. Donoho, and J. M. Pauly, “Sparse mri: The application of compressed sensing for rapid mr imaging,” Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine, vol. 58, no. 6, pp. 1182–1195, 2007.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets