Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Score-Based Generative Models for PET Image Reconstruction (2308.14190v2)

Published 27 Aug 2023 in eess.IV, cs.AI, cs.CV, and cs.LG

Abstract: Score-based generative models have demonstrated highly promising results for medical image reconstruction tasks in magnetic resonance imaging or computed tomography. However, their application to Positron Emission Tomography (PET) is still largely unexplored. PET image reconstruction involves a variety of challenges, including Poisson noise with high variance and a wide dynamic range. To address these challenges, we propose several PET-specific adaptations of score-based generative models. The proposed framework is developed for both 2D and 3D PET. In addition, we provide an extension to guided reconstruction using magnetic resonance images. We validate the approach through extensive 2D and 3D $\textit{in-silico}$ experiments with a model trained on patient-realistic data without lesions, and evaluate on data without lesions as well as out-of-distribution data with lesions. This demonstrates the proposed method's robustness and significant potential for improved PET reconstruction.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (75)
  1. Learned primal-dual reconstruction. IEEE Transactions on Medical Imaging, 37(6):1322–1332, 2018.
  2. A globally convergent image reconstruction for emission tomography using relaxed ordered subsets algorithms. IEEE Transactions on Medical Imaging, 22(5):623–626, 2003.
  3. Brian D O Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Applications, 12(3):313–326, 1982.
  4. On instabilities of deep learning in image reconstruction and the potential costs of ai. Proceedings of the National Academy of Sciences, 117(48):30088–30095, 2020.
  5. Twenty new digital brain phantoms for creation of validation image data bases. IEEE Transactions on Medical Imaging, 25(11):1410–1416, 2006.
  6. Magnetic resonance-guided positron emission tomography image reconstruction. In Seminars in Nuclear Medicine, volume 43, pages 30–44. Elsevier, 2013.
  7. Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8):1798–1828, 2013.
  8. Image deblurring with Poisson data: from cells to galaxies. Inverse Problems, 25(12):123006, 26, 2009. ISSN 0266-5611,1361-6420. . URL https://doi.org/10.1088/0266-5611/25/12/123006.
  9. Score-based diffusion models for accelerated MRI. Medical Image Analysis, 80:102479, 2022.
  10. Diffusion posterior sampling for general noisy inverse problems. In Proceedings of the Eleventh International Conference on Learning Representations. ICLR, 2023a.
  11. Fast diffusion sampler for inverse problems by geometric decomposition. CoRR, abs/2303.05754, 2023b.
  12. Solving 3D inverse problems using pre-trained 2D diffusion models. In Proceedings of the Thirty Seventh IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22542–22551. CVPR, 2023c.
  13. Design and construction of a realistic digital brain phantom. IEEE Transactions on Medical Imaging, 17(3):463–468, 1998.
  14. Test-time training can close the natural distribution shift performance gap in deep learning based compressed sensing. In International Conference on Machine Learning, pages 4754–4776. PMLR, 2022.
  15. Diffusion models beat GANs on image synthesis. In Proceedings of the Thirty Fourth Conference on Advances in Neural Information Processing Systems, pages 8780–8794. NeurIPS, 2021.
  16. Alexandros G Dimakis. Deep generative models and inverse problems. In Mathematical Aspects of Deep Learning, page 400–421. Cambridge University Press, Cambridge, UK, 2022.
  17. Bradley Efron. Tweedie’s formula and selection bias. Journal of the American Statistical Association, 106(496):1602–1614, 2011.
  18. Matthias J Ehrhardt. Multi-modality imaging with structure-promoting regularizers. In Ke Chen, Carola-Bibiane Schönlieb, Xue-Cheng Tai, and Laurent Younces, editors, Handbook of Mathematical Models and Algorithms in Computer Vision and Imaging: Mathematical Imaging and Vision, pages 1–38. Springer International Publishing, Cham, Switzerland, 2021. ISBN 978-3-030-03009-4.
  19. PET reconstruction with an anatomical MRI prior using parallel level sets. IEEE Transactions on Medical Imaging, 35(9):2189–2199, 2016.
  20. Regularization of inverse problems. Mathematics and its Applications. Kluwer Academic Publishers Group, Dordrecht, 1996.
  21. PET image reconstruction using deep image prior. IEEE Transactions on Medical Imaging, 38(7):1655–1665, 2019.
  22. PET image denoising based on denoising diffusion probabilistic models. CoRR, abs/2209.06167, 2022.
  23. Generative adversarial nets. In Proceedings of the Second Conference on Advances in Neural Information Processing Systems, pages 2672–2680. NeurIPS, 2014.
  24. Learned primal dual reconstruction for PET. Journal of Imaging, 7(12), 2021.
  25. Algebraic reconstruction techniques can be made computationally efficient (positron emission tomography application). IEEE Transactions on Medical Imaging, 12(3):600–609, 1993.
  26. Classifier-free diffusion guidance. CoRR, abs/2207.12598, 2022.
  27. Denoising diffusion probabilistic models. In Proceedings of the Thirty Third Conference on Advances in Neural Information Processing Systems. NeurIPS, 2020.
  28. Inverse problems with Poisson data: statistical regularization theory, applications and algorithms. Inverse Problems, 32(9):093001, 56, 2016. ISSN 0266-5611,1361-6420. . URL https://doi.org/10.1088/0266-5611/32/9/093001.
  29. Accelerated image reconstruction using ordered subsets of projection data. IEEE Transactions on Medical Imaging, 13(4):601–609, 1994.
  30. Inverse problems: Tikhonov theory and algorithms. World Scientific, Hackensack, NJ, 2015. ISBN 978-981-4596-19-0.
  31. Robust compressed sensing MRI with deep generative priors. In Proceedings of the Thirty Fourth Conference on Advances in Neural Information Processing Systems, pages 14938–14954. NeurIPS, 2021.
  32. Deep convolutional neural network for inverse problems in imaging. IEEE Transactions on Image Processing, 26(9):4509–4522, 2017.
  33. Full-dose PET image estimation from low-dose PET image using deep learning: a pilot study. Journal of Digital Imaging, 32(5):773–778, 2019.
  34. Quantitative comparison of PET performance—siemens biograph mCT and mMR. European Journal of Nuclear Medicine and Molecular Imaging: Physics, 3(1), 2016.
  35. Auto-encoding variational bayes. In Proceedings of the Second International Conference on Learning Representations. ICLR, 2014.
  36. Learning gradually non-convex image priors using score matching. CoRR, abs/2302.10502, 2023.
  37. DIVα𝛼\alphaitalic_αl library, 2021. URL https://doi.org/10.5281/zenodo.4428220.
  38. MRI reconstruction with side information using diffusion models. arXiv, 2023. URL https://arxiv.org/abs/2303.14795.
  39. Diffusion-LM improves controllable text generation. In Proceedings to the Thirty Fifth Conference on Advances in Neural Information Processing Systems, pages 4328–4343. NeurIPS, 2022.
  40. Reflected diffusion models. In Proceedings of the Fortieth International Conference on Machine Learning, volume 202, pages 22675–22701. PMLR, 2023.
  41. Model-based deep learning PET image reconstruction using forward–backward splitting expectation–maximization. IEEE Transactions on Radiation and Plasma Medical Sciences, 5(1):54–64, 2021.
  42. A concave prior penalizing relative differences for maximum-a-posteriori reconstruction in emission tomography. IEEE Transactions on Nuclear Science, 49(1):56–60, 2002.
  43. List-mode PET image reconstruction using deep image prior. IEEE Transactions on Medical Imaging, 42(6):1822–1834, 2023.
  44. SIRF: synergistic image reconstruction framework. Computer Physics Communications, 249:107087, 2020.
  45. Deep learning-based image reconstruction and post-processing methods in positron emission tomography for low-dose imaging and resolution enhancement. European Journal of Nuclear Medicine and Molecular Imaging, 49(9):3098–3118, 2022.
  46. Alvaro R De Pierro and Michel E B Yamagishi. Fast EM-like methods for maximum ‘a posteriori’ estimates in emission tomography. IEEE Transactions on Medical Imaging, 20(4):280–288, 2001.
  47. Brain imaging generation with latent diffusion models. In Proceedings of the Twenty Fifth Conference on Medical Image Computing and Computer Assisted Intervention, pages 117–126. MICCAI, 2022.
  48. Iterative reconstruction techniques in emission computed tomography. Physics in Medicine & Biology, 51(15):R541, 2006.
  49. Denoising score-matching for uncertainty quantification in inverse problems. CoRR, abs/2011.08698, 2020.
  50. Optimizing sampling patterns for compressed sensing MRI with diffusion generative models. CoRR, abs/2306.03284, 2023.
  51. Deep learning for PET image reconstruction. IEEE Transactions on Radiation and Plasma Medical Sciences, 5(1):1–25, 2021.
  52. U-net: Convolutional networks for biomedical image segmentation. In Proceedings of the Eighteenth Conference on Medical Image Computing and Computer-Assisted Intervention, pages 234–241. MICCAI, 2015.
  53. Nonlinear total variation based noise removal algorithms. Physica D: Nonlinear Phenomena, 60(1-4):259–268, 1992.
  54. Photorealistic text-to-image diffusion models with deep language understanding. In Proceedings of the Thirty Fifth Conference on Advances in Neural Information Processing Systems, pages 36479–36494. NeurIPS, 2022.
  55. Georg Schramm. Simulated brainweb PET/MR data sets for denoising and deblurring, 2021. URL https://zenodo.org/record/4897350.
  56. Georg Schramm. PARALLELPROJ – an open-source framework for fast calculation of projections in tomography, 2022. URL https://arxiv.org/abs/2212.12519.
  57. Maximum likelihood reconstruction for emission tomography. IEEE Transactions on Medical Imaging, 1(2):113–122, 1982.
  58. 3D PET-DIP reconstruction with relative difference prior using a SIRF-based objective. In Proceedings to the Seventeenth International Meeting on Fully Three-Dimensional Image Reconstruction in Radiology and Nuclear Medicine. Fully3D, 2023.
  59. Investigating intensity normalisation for pet reconstruction with supervised deep learning. In Proceedings of 2023 IEEE Nuclear Science Symposium and Medical Imaging Conference, together with the International Symposium on Room-Temperature Semiconductor X-Ray and Gamma-Ray Detectors, in-press. URL https://discovery.ucl.ac.uk/id/eprint/10181467.
  60. Deep unsupervised learning using nonequilibrium thermodynamics. In Proceedings of the Thirty Second International Conference on Machine Learning, pages 2256–2265, 2015.
  61. PET image reconstruction using information theoretic anatomical priors. IEEE Transactions on Medical Imaging, 30(3):537–549, 2011.
  62. Denoising diffusion implicit models. In Proceedings of the Ninth International Conference on Learning Representations. ICLR, 2021a.
  63. Generative modeling by estimating gradients of the data distribution. In Proceedings of the Thirty Second Conference on Advances in Neural Information Processing Systems, pages 11895–11907. NeurIPS, 2019.
  64. Maximum likelihood training of score-based diffusion models. In Proceedings of the Thirty Fourth Conference on Advances in Neural Information Processing Systems, pages 1415–1428. NeurIPS, 2021b.
  65. Score-based generative modeling through stochastic differential equations. In Proceedings of the Ninth International Conference on Learning Representations. ICLR, 2021c.
  66. Solving inverse problems in medical imaging with score-based generative models. In Proceedings of the Tenth International Conference on Learning Representations. ICLR, 2022.
  67. Applied Stochastic Differential Equations. Institute of Mathematical Statistics Textbooks. Cambridge University Press, Cambridge, 2019. ISBN 978-131-6649-46-6.
  68. STIR: software for tomographic image reconstruction release 2. Physics in Medicine & Biology, 57(4):867, 2012.
  69. Noise and signal properties in PSF-based fully 3D PET image reconstruction: an experimental evaluation. Physics in Medicine & Biology, 55(5):1453, 2010.
  70. Practical uncertainty estimation and out-of-distribution robustness in deep learning. NeurIPS Tutorial, Google Brain, 2020. URL https://neurips.cc/media/neurips-2020/Slides/16649.pdf.
  71. Deep image prior. In Proceedings of the Thirty First IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9446–9454. CVPR, 2018.
  72. Pascal Vincent. A connection between score matching and denoising autoencoders. Neural Computation, 23(7):1661–1674, 2011.
  73. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600–612, 2004.
  74. Tackling the generative learning trilemma with denoising diffusion GANs. In Proceedings of the Tenth International Conference on Learning Representations. ICLR, 2022.
  75. Denoising diffusion models for plug-and-play image restoration. CoRR, abs/2305.08995, 2023.
Citations (9)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now