Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Self-Consistent Recursive Diffusion Bridge for Medical Image Translation (2405.06789v1)

Published 10 May 2024 in eess.IV and cs.CV

Abstract: Denoising diffusion models (DDM) have gained recent traction in medical image translation given improved training stability over adversarial models. DDMs learn a multi-step denoising transformation to progressively map random Gaussian-noise images onto target-modality images, while receiving stationary guidance from source-modality images. As this denoising transformation diverges significantly from the task-relevant source-to-target transformation, DDMs can suffer from weak source-modality guidance. Here, we propose a novel self-consistent recursive diffusion bridge (SelfRDB) for improved performance in medical image translation. Unlike DDMs, SelfRDB employs a novel forward process with start- and end-points defined based on target and source images, respectively. Intermediate image samples across the process are expressed via a normal distribution with mean taken as a convex combination of start-end points, and variance from additive noise. Unlike regular diffusion bridges that prescribe zero variance at start-end points and high variance at mid-point of the process, we propose a novel noise scheduling with monotonically increasing variance towards the end-point in order to boost generalization performance and facilitate information transfer between the two modalities. To further enhance sampling accuracy in each reverse step, we propose a novel sampling procedure where the network recursively generates a transient-estimate of the target image until convergence onto a self-consistent solution. Comprehensive analyses in multi-contrast MRI and MRI-CT translation indicate that SelfRDB offers superior performance against competing methods.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (73)
  1. J. E. Iglesias, E. Konukoglu, D. Zikic, B. Glocker, K. Van Leemput, and B. Fischl, “Is synthesizing MRI contrast useful for inter-modality analysis?” in Med Image Comput Comput Assist Interv, 2013, pp. 631–638.
  2. J. Lee, A. Carass, A. Jog, C. Zhao, and J. Prince, “Multi-atlas-based CT synthesis from conventional MRI with patch-based refinement for MRI-based radiotherapy planning,” in SPIE Med Imag., vol. 10133, 2017, p. 101331I.
  3. D. H. Ye, D. Zikic, B. Glocker, A. Criminisi, and E. Konukoglu, “Modality propagation: Coherent synthesis of subject-specific scans with data-driven regularization,” in Med Image Comput Comput Assist Interv, 2013, pp. 606–613.
  4. T. Huynh, Y. Gao, J. Kang, L. Wang, P. Zhang, J. Lian, and D. Shen, “Estimating CT image from MRI data using structured random forest and auto-context model,” IEEE Trans Med Imag, vol. 35, no. 1, pp. 174–183, 2016.
  5. A. Jog, A. Carass, S. Roy, D. L. Pham, and J. L. Prince, “Random forest regression for magnetic resonance image synthesis,” Med Image Anal, vol. 35, pp. 475–488, 2017.
  6. T. Joyce, A. Chartsias, and S. A. Tsaftaris, “Robust multi-modal MR image synthesis,” in Med Image Comput Comput Assist Interv, 2017, pp. 347–355.
  7. N. Cordier, H. Delingette, M. Le, and N. Ayache, “Extended modality propagation: Image synthesis of pathological cases,” IEEE Trans Med Imag, vol. 35, pp. 2598–2608, 2016.
  8. Y. Wu, W. Yang, L. Lu, Z. Lu, L. Zhong, M. Huang, Y. Feng, Q. Feng, and W. Chen, “Prediction of CT substitutes from MR images based on local diffeomorphic mapping for brain PET attenuation correction,” J Nucl Med, vol. 57, no. 10, pp. 1635–1641, 2016.
  9. C. Zhao, A. Carass, J. Lee, Y. He, and J. L. Prince, “Whole brain segmentation and labeling from CT using synthetic MR images,” in Mach Learn Med Imaging, 2017, pp. 291–298.
  10. Y. Huang, L. Shao, and A. F. Frangi, “Cross-modality image synthesis via weakly coupled and geometry co-regularized joint dictionary learning,” IEEE Trans Med Imag, vol. 37, no. 3, pp. 815–827, 2018.
  11. D. Lee, J. Kim, W.-J. Moon, and J. C. Ye, “CollaGAN: Collaborative GAN for missing image data imputation,” in Comput Vis Pattern Recognit, 2019, pp. 2487–2496.
  12. L. T. Clark et al., “Increasing Diversity in Clinical Trials: Overcoming Critical Barriers,” Cur. Prob. Cardiol., vol. 44, no. 5, pp. 148–172, 2019.
  13. S. Roy, A. Jog, A. Carass, and J. L. Prince, “Atlas based intensity transformation of brain MR images,” in Multimodal Brain Image Anal., 2013, pp. 51–62.
  14. D. C. Alexander, D. Zikic, J. Zhang, H. Zhang, and A. Criminisi, “Image quality transfer via random forest regression: Applications in diffusion MRI,” in Med Image Comput Comput Assist Interv, 2014, pp. 225–232.
  15. Y. Huang, L. Shao, and A. F. Frangi, “Simultaneous super-resolution and cross-modality synthesis of 3D medical images using weakly-supervised joint convolutional sparse coding,” Comput Vis Pattern Recognit, pp. 5787–5796, 2017.
  16. H. Van Nguyen, K. Zhou, and R. Vemulapalli, “Cross-domain synthesis of medical images using efficient location-sensitive deep network,” in Med Image Comput Comput Assist Interv, 2015, pp. 677–684.
  17. R. Vemulapalli, H. Van Nguyen, and S. K. Zhou, “Unsupervised cross-modal synthesis of subject-specific scans,” in Int Conf Comput Vis, 2015, pp. 630–638.
  18. V. Sevetlidis, M. V. Giuffrida, and S. A. Tsaftaris, “Whole image synthesis using a deep encoder-decoder network,” in Simul Synth Med Imaging, 2016, pp. 127–137.
  19. D. Nie, X. Cao, Y. Gao, L. Wang, and D. Shen, “Estimating CT image from MRI data using 3D fully convolutional networks,” in Deep Learn Data Label Med Appl, 2016, pp. 170–178.
  20. C. Bowles et al., “Pseudo-healthy image synthesis for white matter lesion segmentation,” in Simul Synth Med Imaging, 2016, pp. 87–96.
  21. A. Chartsias, T. Joyce, M. V. Giuffrida, and S. A. Tsaftaris, “Multimodal MR synthesis via modality-invariant latent representation,” IEEE Trans Med Imag, vol. 37, no. 3, pp. 803–814, 2018.
  22. W. Wei, E. Poirion, B. Bodini, S. Durrleman, O. Colliot, B. Stankoff, and N. Ayache, “Fluid-attenuated inversion recovery MRI synthesis from multisequence MRI using three-dimensional fully convolutional networks for multiple sclerosis,” J Med Imaging, vol. 6, no. 1, p. 014005, 2019.
  23. B. Yu, L. Zhou, L. Wang, J. Fripp, and P. Bourgeat, “3D cGAN based cross-modality MR image synthesis for brain tumor segmentation,” Int. Symp. Biomed. Imaging, pp. 626–630, 2018.
  24. K. Armanious, C. Jiang, M. Fischer, T. Küstner, T. Hepp, K. Nikolaou, S. Gatidis, and B. Yang, “MedGAN: Medical image translation using GANs,” Comput Med Imaging Grap, vol. 79, p. 101684, 2019.
  25. H. Li, J. C. Paetzold, A. Sekuboyina, F. Kofler, J. Zhang, J. S. Kirschke, B. Wiestler, and B. Menze, “DiamondGAN: Unified multi-modal generative adversarial networks for MRI sequences synthesis,” in Med. Image Comput Comput Assist Interv, 2019, pp. 795–803.
  26. D. Nie, R. Trullo, J. Lian, L. Wang, C. Petitjean, S. Ruan, and Q. Wang, “Medical image synthesis with deep convolutional adversarial networks,” IEEE Trans. Biomed. Eng., vol. 65, no. 12, pp. 2720–2730, 2018.
  27. S. U. Dar, M. Yurt, L. Karacan, A. Erdem, E. Erdem, and T. Cukur, “Image synthesis in multi-contrast MRI with conditional generative adversarial networks,” IEEE Trans Med Imaging, vol. 38, no. 10, pp. 2375–2388, 2019.
  28. B. Yu, L. Zhou, L. Wang, Y. Shi, J. Fripp, and P. Bourgeat, “Ea-GANs: Edge-aware generative adversarial networks for cross-modality MR image synthesis,” IEEE Trans Med Imag, vol. 38, no. 7, pp. 1750–1762, 2019.
  29. H. Yang, J. Sun, A. Carass, C. Zhao, J. Lee, Z. Xu, and J. Prince, “Unpaired brain MR-to-CT synthesis using a structure-constrained cycleGAN,” arXiv:1809.04536, 2018.
  30. C.-B. Jin, H. Kim, M. Liu, W. Jung, S. Joo, E. Park, Y. S. Ahn, I. H. Han, J. I. Lee, and X. Cui, “Deep CT to MR synthesis using paired and unpaired data,” Sensors, vol. 19, no. 10, p. 2361, 2019.
  31. O. Dalmaz, M. Yurt, and T. Çukur, “ResViT: Residual vision transformers for multi-modal medical image synthesis,” IEEE Trans Med Imaging, vol. 44, no. 10, pp. 2598–2614, 2022.
  32. G. Wang, E. Gong, S. Banerjee, D. Martin, E. Tong, J. Choi, H. Chen, M. Wintermark, J. M. Pauly, and G. Zaharchuk, “Synthesize high-quality multi-contrast magnetic resonance imaging from multi-echo acquisition using multi-task deep generative model,” IEEE Trans Med Imag, vol. 39, no. 10, pp. 3089–3099, 2020.
  33. T. Zhou, H. Fu, G. Chen, J. Shen, and L. Shao, “Hi-Net: Hybrid-fusion network for multi-modal MR image synthesis,” IEEE Trans Med Imag, vol. 39, no. 9, pp. 2772–2781, 2020.
  34. M. Özbey, S. U. Dar, H. A. Bedel, O. Dalmaz, Ş. Özturk, A. Güngör, and T. Çukur, “Unsupervised medical image translation with adversarial diffusion models,” IEEE Trans Med Imaging, vol. 42, no. 12, pp. 3524–3539, 2023.
  35. X. Meng, Y. Gu, Y. Pan, N. Wang, P. Xue, M. Lu, X. He, Y. Zhan, and D. Shen, “A novel unified conditional score-based generative framework for multi-modal medical image completion,” arXiv:2207.03430, 2022.
  36. Q. Lyu and G. Wang, “Conversion between CT and MRI images using diffusion and score-matching models,” arXiv:2209.12104, 2022.
  37. Z. Wang, Y. Yang, Y. Chen, T. Yuan, M. Sermesant, H. Delingette, and O. Wu, “Mutual information guided diffusion for zero-shot cross-modality medical image translation,” IEEE Trans Med Imaging, pp. 1–1, 2024.
  38. Y. Song, L. Shen, L. Xing, and S. Ermon, “Solving inverse problems in medical imaging with score-based generative models,” arXiv:2111.08005, 2021.
  39. A. Güngör, S. U. Dar, Şaban Öztürk, Y. Korkmaz, H. A. Bedel, G. Elmas, M. Ozbey, and T. Çukur, “Adaptive diffusion priors for accelerated mri reconstruction,” Med Image Anal, p. 102872, 2023.
  40. J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” in Adv Neural Inf Process Syst, vol. 33, 2020, pp. 6840–6851.
  41. M. Delbracio and P. Milanfar, “Inversion by direct iteration: An alternative to denoising diffusion for image restoration,” Tran. Mach. Learn. Res., 2023.
  42. H. Chung, J. Kim, and J. C. Ye, “Direct diffusion bridge using data consistency for inverse problems,” arXiv:2305.19809, 2023.
  43. G.-H. Liu, A. Vahdat, D.-A. Huang, E. A. Theodorou, W. Nie, and A. Anandkumar, “I2SB: Image-to-Image Schrödinger Bridge,” arXiv:2302.05872, 2023.
  44. B. Kim, G. Kwon, K. Kim, and J. C. Ye, “Unpaired image-to-image translation via neural schrödinger bridge,” in ICLR, 2024.
  45. M. U. Mirza, O. Dalmaz, H. A. Bedel, G. Elmas, Y. Korkmaz, A. Gungor, S. U. Dar, and T. Çukur, “Learning Fourier-Constrained Diffusion Bridges for MRI Reconstruction,” arXiv:2308.01096, 2023.
  46. J. Kim and J. C. Ye, “HiCBridge: Resolution enhancement of hi-c data using direct diffusion bridge,” 2024. [Online]. Available: https://openreview.net/forum?id=RUvzlotXY0
  47. X. Su, J. Song, C. Meng, and S. Ermon, “Dual diffusion implicit bridges for image-to-image translation,” arXic:2203.08382, 2023.
  48. C. Peng, P. Guo, S. K. Zhou, V. Patel, and R. Chellappa, “Towards performant and reliable undersampled mr reconstruction via diffusion model sampling,” arXiv:2203.04292, 2022.
  49. J. M. Wolterink, A. M. Dinkla, M. H. F. Savenije, P. R. Seevinck, C. A. T. van den Berg, and I. Išgum, “Deep MR to CT synthesis using unpaired data,” in Simul Synth Med Imaging, Cham, 2017, pp. 14–23.
  50. X. Dong, T. Wang, Y. Lei, K. Higgins, T. Liu, W. Curran, H. Mao, J. Nye, and X. Yang, “Synthetic CT generation from non-attenuation corrected PET images for whole-body PET imaging,” Phys Med Biol, vol. 64, no. 21, p. 215016, 2019.
  51. G. Daras, M. Delbracio, H. Talebi, A. G. Dimakis, and P. Milanfar, “Soft diffusion: Score matching for general corruptions,” arXiv:2209.05442, 2022.
  52. T. Chen, G.-H. Liu, and E. A. Theodorou, “Likelihood training of schr\\\backslash\” odinger bridge using forward-backward sdes theory,” arXiv preprint arXiv:2110.11291, 2021.
  53. Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, “Score-based generative modeling through stochastic differential equations,” arXiv:2011.13456, 2020.
  54. Z. Chen, G. He, K. Zheng, X. Tan, and J. Zhu, “Schrodinger bridges beat diffusion models on text-to-speech synthesis,” arXiv preprint arXiv:2312.03491, 2023.
  55. Y. Song and S. Ermon, “Generative modeling by estimating gradients of the data distribution,” Advances in neural information processing systems, vol. 32, 2019.
  56. Z. Xiao, K. Kreis, and A. Vahdat, “Tackling the generative learning trilemma with denoising diffusion GANs,” in Int Conf Learn Represent, 2022.
  57. S. U. Dar, M. Yurt, L. Karacan, A. Erdem, E. Erdem, and T. Çukur, “Image synthesis in multi-contrast MRI with conditional generative adversarial networks,” IEEE Trans Med Imag, vol. 38, no. 10, pp. 2375–2388, 2019.
  58. B. H. Menze et al., “The multimodal brain tumor image segmentation benchmark (BRATS),” IEEE Trans Med Imag, vol. 34, no. 10, pp. 1993–2024, 2015.
  59. T. Nyholm et al., “MR and CT data with multiobserver delineations of organs in the pelvic area—part of the gold atlas project,” Med Phys, vol. 45, no. 3, pp. 1295–1300, 2018.
  60. M. Jenkinson and S. Smith, “A global optimisation method for robust affine registration of brain images,” Med Image Anal, vol. 5, pp. 143–156, 2001.
  61. O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in Med Image Comput Comput Assist Inter.   Springer, 2015, pp. 234–241.
  62. A. Q. Nichol and P. Dhariwal, “Improved denoising diffusion probabilistic models,” in Int Conf Mach Learn, 2021, pp. 8162–8171.
  63. H. Lan, A. Toga, and F. Sepehrband, “SC-GAN: 3D self-attention conditional GAN with spectral normalization for multi-modal neuroimaging synthesis,” bioRxiv:2020.06.09.143297, 2020.
  64. Y. Ge, D. Wei, Z. Xue, Q. Wang, X. Zhou, Y. Zhan, and S. Liao, “Unpaired MR to CT synthesis with explicit structural constrained adversarial learning,” in Int. Symp. Biomed. Imaging, 2019, pp. 1096–1099.
  65. G. Elmas, S. U. Dar, Y. Korkmaz, E. Ceyani, B. Susam, M. Özbey, S. Avestimehr, and T. Çukur, “Federated Learning of Generative Image Priors for MRI Reconstruction,” IEEE Trans Med Imaging, vol. 42, no. 7, pp. 1996–2009, 2023.
  66. A. Sharma and G. Hamarneh, “Missing MRI pulse sequence synthesis using multi-modal generative adversarial network,” IEEE Trans Med Imag, vol. 39, pp. 1170–1183, 2020.
  67. M. Yurt, S. U. Dar, A. Erdem, E. Erdem, K. K. Oguz, and T. Çukur, “mustGAN: multi-stream generative adversarial networks for MR image synthesis,” Med Image Anal, vol. 70, p. 101944, 2021.
  68. Y. Luo, Y. Wang, C. Zu, B. Zhan, X. Wu, J. Zhou, D. Shen, and L. Zhou, “3D Transformer-GAN for high-quality PET reconstruction,” in Med Image Comput Comput Assist Interv, 2021, pp. 276–285.
  69. A. Gungor, B. Askin, D. A. Soydan, E. U. Saritas, C. B. Top, and T. Çukur, “TranSMS: Transformers for super-resolution calibration in magnetic particle imaging,” IEEE Trans Med Imaging, vol. 41, no. 12, pp. 3562–3574, 2022.
  70. Y. Korkmaz, S. U. H. Dar, M. Yurt, M. Ozbey, and T. Cukur, “Unsupervised MRI reconstruction via zero-shot learned adversarial transformers,” IEEE Trans Med Imaging, vol. 41, no. 7, pp. 1747–1763, 2022.
  71. Y. Korkmaz, T. Cukur, and V. M. Patel, “Self-supervised mri reconstruction with unrolled diffusion models,” in MICCAI, 2023, pp. 491–501.
  72. H. Chung, B. Sim, and J. C. Ye, “Come-closer-diffuse-faster: Accelerating conditional diffusion models for inverse problems through stochastic contraction,” in IEEE Conf Comput Vis Pattern Recognit, 2022, pp. 12 413–12 422.
  73. H. A. Bedel and T. Çukur, “DreaMR: Diffusion-driven counterfactual explanation for functional MRI,” arXiv:2307.09547, 2023.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Fuat Arslan (4 papers)
  2. Bilal Kabas (6 papers)
  3. Onat Dalmaz (10 papers)
  4. Tolga Çukur (48 papers)
  5. Muzaffer Ozbey (1 paper)
Citations (5)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets