MVMS-RCN: A Dual-Domain Unfolding CT Reconstruction with Multi-sparse-view and Multi-scale Refinement-correction (2405.17141v2)
Abstract: X-ray Computed Tomography (CT) is one of the most important diagnostic imaging techniques in clinical applications. Sparse-view CT imaging reduces the number of projection views to a lower radiation dose and alleviates the potential risk of radiation exposure. Most existing deep learning (DL) and deep unfolding sparse-view CT reconstruction methods: 1) do not fully use the projection data; 2) do not always link their architecture designs to a mathematical theory; 3) do not flexibly deal with multi-sparse-view reconstruction assignments. This paper aims to use mathematical ideas and design optimal DL imaging algorithms for sparse-view tomography reconstructions. We propose a novel dual-domain deep unfolding unified framework that offers a great deal of flexibility for multi-sparse-view CT reconstruction with different sampling views through a single model. This framework combines the theoretical advantages of model-based methods with the superior reconstruction performance of DL-based methods, resulting in the expected generalizability of DL. We propose a refinement module that utilizes unfolding projection domain to refine full-sparse-view projection errors, as well as an image domain correction module that distills multi-scale geometric error corrections to reconstruct sparse-view CT. This provides us with a new way to explore the potential of projection information and a new perspective on designing network architectures. All parameters of our proposed framework are learnable end to end, and our method possesses the potential to be applied to plug-and-play reconstruction. Extensive experiments demonstrate that our framework is superior to other existing state-of-the-art methods. Our source codes are available at https://github.com/fanxiaohong/MVMS-RCN.
- X. Pan, E. Y. Sidky, and M. Vannier, “Why do commercial CT scanners still employ traditional, filtered back-projection for image reconstruction?” Inverse Problems, vol. 25, no. 12, p. 123009, Dec. 2009.
- S. Abbas, T. Lee, S. Shin, R. Lee, and S. Cho, “Effects of sparse sampling schemes on image quality in low-dose CT,” Medical Physics, vol. 40, no. 11, p. 111915, Oct. 2013.
- K. Kim, J. C. Ye, W. Worstell, J. Ouyang, Y. Rakvongthai, G. E. Fakhri, and Q. Li, “Sparse-view spectral CT reconstruction using spectral patch-based low-rank penalty,” IEEE Transactions on Medical Imaging, vol. 34, no. 3, pp. 748–760, Mar. 2015.
- A. K. Louis, “Incomplete data problems in x-ray computerized tomography,” Numerische Mathematik, vol. 48, no. 3, pp. 251–262, May 1986.
- S. J. LaRoque, E. Y. Sidky, and X. Pan, “Accurate image reconstruction from few-view and limited-angle data in diffraction tomography,” Journal of the Optical Society of America A, vol. 25, no. 7, p. 1772, Jun. 2008.
- J. Yang, H. Yu, M. Jiang, and G. Wang, “High-order total variation minimization for interior tomography,” Inverse Problems, vol. 26, no. 3, p. 035013, Feb. 2010.
- D. Zeng, J. Huang, H. Zhang, Z. Bian, S. Niu, Z. Zhang, Q. Feng, W. Chen, and J. Ma, “Spectral CT image restoration via an average image-induced nonlocal means filter,” IEEE Transactions on Biomedical Engineering, vol. 63, no. 5, pp. 1044–1057, May 2016.
- M. Rantala, S. Vanska, S. Jarvenpaa, M. Kalke, M. Lassas, J. Moberg, and S. Siltanen, “Wavelet-based reconstruction for limited-angle x-ray tomography,” IEEE Transactions on Medical Imaging, vol. 25, no. 2, pp. 210–217, Feb. 2006.
- B. Dong, J. Li, and Z. Shen, “X-ray CT image reconstruction via wavelet frame based regularization and radon domain inpainting,” Journal of Scientific Computing, vol. 54, no. 2-3, pp. 333–349, Feb. 2013.
- P. Bao, H. Sun, Z. Wang, Y. Zhang, W. Xia, K. Yang, W. Chen, M. Chen, Y. Xi, S. Niu, J. Zhou, and H. Zhang, “Convolutional sparse coding for compressed sensing CT reconstruction,” IEEE Transactions on Medical Imaging, vol. 38, no. 11, pp. 2607–2619, Nov. 2019.
- S. Lu, B. Yang, Y. Xiao, S. Liu, M. Liu, L. Yin, and W. Zheng, “Iterative reconstruction of low-dose CT based on differential sparse,” Biomedical Signal Processing and Control, vol. 79, p. 104204, Jan. 2023.
- H. Gao, J. Cai, Z. Shen, and H. Zhao, “Robust principal component analysis-based four-dimensional computed tomography,” Physics in Medicine and Biology, vol. 56, no. 11, pp. 3181–3198, May 2011.
- O. Semerci, N. Hao, M. E. Kilmer, and E. L. Miller, “Tensor-based formulation and nuclear norm regularization for multienergy computed tomography,” IEEE Transactions on Image Processing, vol. 23, no. 4, pp. 1678–1693, Apr. 2014.
- B. Zhao, H. Ding, Y. Lu, G. Wang, J. Zhao, and S. Molloi, “Dual-dictionary learning-based iterative image reconstruction for spectral computed tomography application,” Physics in Medicine and Biology, vol. 57, no. 24, pp. 8217–8229, Nov. 2012.
- Y. Zhang, X. Mou, G. Wang, and H. Yu, “Tensor-based dictionary learning for spectral CT reconstruction,” IEEE Transactions on Medical Imaging, vol. 36, no. 1, pp. 142–154, Jan. 2017.
- Y. Dong, P. C. Hansen, and H. M. Kjer, “Joint CT reconstruction and segmentation with discriminative dictionary learning,” IEEE Transactions on Computational Imaging, vol. 4, no. 4, pp. 528–536, Dec. 2018.
- S. Li, Q. Li, R. Li, W. Wu, J. Zhao, Y. Qiang, and Y. Tian, “An adaptive self-guided wavelet convolutional neural network with compound loss for low-dose CT denoising,” Biomedical Signal Processing and Control, vol. 75, p. 103543, may 2022.
- H. Chen, Y. Zhang, M. K. Kalra, F. Lin, Y. Chen, P. Liao, J. Zhou, and G. Wang, “Low-dose CT with a residual encoder-decoder convolutional neural network,” IEEE Transactions on Medical Imaging, vol. 36, no. 12, pp. 2524–2535, Dec. 2017.
- K. H. Jin, M. T. McCann, E. Froustey, and M. Unser, “Deep convolutional neural network for inverse problems in imaging,” IEEE Transactions on Image Processing, vol. 26, no. 9, pp. 4509–4522, Sep. 2017.
- K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770–778.
- Z. Zhang, X. Liang, X. Dong, Y. Xie, and G. Cao, “A sparse-view CT reconstruction method based on combination of DenseNet and deconvolution,” IEEE Transactions on Medical Imaging, vol. 37, no. 6, pp. 1407–1417, Jun. 2018.
- G. Huang, Z. Liu, L. V. D. Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jul. 2017, pp. 2261–2269.
- O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer Assisted Intervention – MICCAI, 2015, pp. 234–241.
- Y. Han and J. C. Ye, “Framing U-net via deep convolutional framelets: Application to sparse-view CT,” IEEE Transactions on Medical Imaging, vol. 37, no. 6, pp. 1418–1429, Jun. 2018.
- Z. Huang, J. Zhang, Y. Zhang, and H. Shan, “DU-GAN: Generative adversarial networks with dual-domain u-net-based discriminators for low-dose CT denoising,” IEEE Transactions on Instrumentation and Measurement, vol. 71, pp. 1–12, 2022.
- D. Wang, F. Fan, Z. Wu, R. Liu, F. Wang, and H. Yu, “CTformer: convolution-free token2token dilated vision transformer for low-dose CT denoising,” Physics in Medicine and Biology, vol. 68, no. 6, p. 065012, Mar. 2023.
- B. Zhou, X. Chen, S. K. Zhou, J. S. Duncan, and C. Liu, “Dudodr-net: Dual-domain data consistent recurrent network for simultaneous sparse view and metal artifact reduction in computed tomography,” Medical Image Analysis, vol. 75, p. 102289, Jan. 2022.
- T. Cheslerean-Boghiu, F. C. Hofmann, M. Schulthei, F. Pfeiffer, D. Pfeiffer, and T. Lasser, “Wnet: A data-driven dual-domain denoising model for sparse-view computed tomography with a trainable reconstruction layer,” IEEE Transactions on Computational Imaging, vol. 9, pp. 120–132, 2023.
- Y. Zhang, T. Lv, R. Ge, Q. Zhao, D. Hu, L. Zhang, J. Liu, Y. Zhang, Q. Liu, W. Zhao, and Y. Chen, “CD-net: Comprehensive domain network with spectral complementary for DECT sparse-view reconstruction,” IEEE Transactions on Computational Imaging, vol. 7, pp. 436–447, Apr. 2021.
- W. Wu, D. Hu, C. Niu, H. Yu, V. Vardhanabhuti, and G. Wang, “DRONE: Dual-domain residual-based optimization network for sparse-view CT reconstruction,” IEEE Transactions on Medical Imaging, vol. 40, no. 11, pp. 3002–3014, Nov. 2021.
- H. Chen, Y. Zhang, Y. Chen, J. Zhang, W. Zhang, H. Sun, Y. Lv, P. Liao, J. Zhou, and G. Wang, “LEARN: Learned experts’ assessment-based reconstruction network for sparse-data CT,” IEEE Transactions on Medical Imaging, vol. 37, no. 6, pp. 1333–1347, Jun. 2018.
- J. Adler and O. Oktem, “Learned primal-dual reconstruction,” IEEE Transactions on Medical Imaging, vol. 37, no. 6, pp. 1322–1332, Jun. 2018.
- J. Zhang and B. Ghanem, “ISTA-Net: Interpretable optimization-inspired deep network for image compressive sensing,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, jun 2018, pp. 1828–1837.
- I. Daubechies, M. Defrise, and C. D. Mol, “An iterative thresholding algorithm for linear inverse problems with a sparsity constraint,” Communications on Pure and Applied Mathematics, vol. 57, no. 11, pp. 1413–1457, 2004.
- M. Elad, “Why simple shrinkage is still relevant for redundant representations?” IEEE Transactions on Information Theory, vol. 52, no. 12, pp. 5559–5569, Dec. 2006.
- J. Xiang, Y. Dong, and Y. Yang, “FISTA-net: Learning a fast iterative shrinkage thresholding network for inverse problems in imaging,” IEEE Transactions on Medical Imaging, vol. 40, no. 5, pp. 1329–1339, May 2021.
- A. Beck and M. Teboulle, “A fast iterative shrinkage-thresholding algorithm for linear inverse problems,” SIAM Journal on Imaging Sciences, vol. 2, no. 1, pp. 183–202, Jan. 2009.
- Z. Zhang, Y. Liu, J. Liu, F. Wen, and C. Zhu, “AMP-Net: Denoising-based deep unfolding for compressive image sensing,” IEEE Transactions on Image Processing, vol. 30, pp. 1487–1500, 2021.
- D. L. Donoho, A. Maleki, and A. Montanari, “Message-passing algorithms for compressed sensing,” in Proceedings of the National Academy of Sciences, vol. 106, no. 45, 2009, pp. 18 914–18 919.
- X. Fan, Y. Yang, K. Chen, Y. Feng, and J. Zhang, “Nest-DGIL: Nesterov-optimized deep geometric incremental learning for cs image reconstruction,” IEEE Transactions on Computational Imaging, vol. 9, pp. 819–833, 2023.
- M. Christopher, M. Ali, and B. Richard, “Learned d-amp: A principled cnn-based compressive image recovery algorithm,” in Advances in Neural Information Processing Systems, 2017.
- M. Borgerding, P. Schniter, and S. Rangan, “AMP-inspired deep networks for sparse linear inverse problems,” IEEE Transactions on Signal Processing, vol. 65, no. 16, pp. 4293–4308, Aug. 2017.
- H. K. Aggarwal, M. P. Mani, and M. Jacob, “MoDL: Model-based deep learning architecture for inverse problems,” IEEE Transactions on Medical Imaging, vol. 38, no. 2, pp. 394–405, Feb. 2019.
- Y. Yang, J. Sun, H. Li, and Z. Xu, “ADMM-CSNet: A deep learning approach for image compressive sensing,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 3, pp. 521–538, Mar. 2020.
- H. Zhang, B. Liu, H. Yu, and B. Dong, “MetaInv-net: Meta inversion network for sparse view CT image reconstruction,” IEEE Transactions on Medical Imaging, vol. 40, no. 2, pp. 621–634, Feb. 2021.
- X. Fan, Y. Yang, and J. Zhang, “Deep geometric distillation network for compressive sensing MRI,” in IEEE EMBS International Conference on Biomedical and Health Informatics (BHI), 2021.
- W. Xia, Z. Lu, Y. Huang, Z. Shi, Y. Liu, H. Chen, Y. Chen, J. Zhou, and Y. Zhang, “MAGIC: Manifold and graph integrative convolutional network for low-dose CT reconstruction,” IEEE Transactions on Medical Imaging, vol. 40, no. 12, pp. 3459–3472, Dec. 2021.
- X. Fan, Y. Yang, K. Chen, J. Zhang, and K. Dong, “An interpretable MRI reconstruction network with two-grid-cycle correction and geometric prior distillation,” Biomedical Signal Processing and Control, vol. 84, p. 104821, Jul. 2023.
- S. F. McCormick, “Multigrid methods,” SIAM, 1987.
- J. Xu, “A new class of iterative methods for nonselfadjoint or indefinite problems,” SIAM Journal on Numerical Analysis, vol. 29, no. 2, pp. 303–319, Apr. 1992.
- J. Xu, “Two-grid discretization techniques for linear and nonlinear PDEs,” SIAM Journal on Numerical Analysis, vol. 33, no. 5, pp. 1759–1777, Oct. 1996.
- P. M. Anselone and J. W. Lee, “Spectral properties of integral operators with nonnegative kernels,” Linear Algebra and its Applications, vol. 9, pp. 67–87, 1974.
- J. Song, B. Chen, and J. Zhang, “Memory-augmented deep unfolding network for compressive sensing,” in Proceedings of the 29th ACM International Conference on Multimedia, Oct. 2021, pp. 4249–4258.
- K. Jiang, Z. Wang, P. Yi, J. Jiang, J. Xiao, and Y. Yao, “Deep distillation recursive network for remote sensing imagery super-resolution,” Remote Sensing, vol. 10, no. 11, p. 1700, Oct. 2018.
- K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification,” in IEEE International Conference on Computer Vision (ICCV), Dec. 2015, pp. 1026–1034.
- T. R. Moen, B. Chen, D. R. Holmes, X. Duan, Z. Yu, L. Yu, S. Leng, J. G. Fletcher, and C. H. McCollough, “Low-dose CT image and projection dataset,” Medical Physics, vol. 48, no. 2, pp. 902–911, Dec. 2020.
- M. Ronchetti, “Torchradon: Fast differentiable routines for computed tomography,” arXiv, 2020.
- K. Diederik and B. Jimmy, “Adam: A method for stochastic optimization,” in Proceedings of ICLR, 2015.
- A. Beck and M. Teboulle, “Fast gradient-based algorithms for constrained total variation image denoising and deblurring problems,” IEEE Transactions on Image Processing, vol. 18, no. 11, pp. 2419–2434, Nov. 2009.
- Z. Wang, X. Cun, J. Bao, W. Zhou, J. Liu, and H. Li, “Uformer: A general u-shaped transformer for image restoration,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2022, pp. 17 662–17 672.
- K. Zhang, Y. Li, W. Zuo, L. Zhang, L. Van Gool, and R. Timofte, “Plug-and-play image restoration with deep denoiser prior,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 10, pp. 6360–6376, 2022.
- Xiaohong Fan (8 papers)
- Ke Chen (241 papers)
- Huaming Yi (1 paper)
- Yin Yang (110 papers)
- Jianping Zhang (49 papers)