Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
144 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

VM-DDPM: Vision Mamba Diffusion for Medical Image Synthesis (2405.05667v1)

Published 9 May 2024 in eess.IV and cs.CV

Abstract: In the realm of smart healthcare, researchers enhance the scale and diversity of medical datasets through medical image synthesis. However, existing methods are limited by CNN local perception and Transformer quadratic complexity, making it difficult to balance structural texture consistency. To this end, we propose the Vision Mamba DDPM (VM-DDPM) based on State Space Model (SSM), fully combining CNN local perception and SSM global modeling capabilities, while maintaining linear computational complexity. Specifically, we designed a multi-level feature extraction module called Multi-level State Space Block (MSSBlock), and a basic unit of encoder-decoder structure called State Space Layer (SSLayer) for medical pathological images. Besides, we designed a simple, Plug-and-Play, zero-parameter Sequence Regeneration strategy for the Cross-Scan Module (CSM), which enabled the S6 module to fully perceive the spatial features of the 2D image and stimulate the generalization potential of the model. To our best knowledge, this is the first medical image synthesis model based on the SSM-CNN hybrid architecture. Our experimental evaluation on three datasets of different scales, i.e., ACDC, BraTS2018, and ChestXRay, as well as qualitative evaluation by radiologists, demonstrate that VM-DDPM achieves state-of-the-art performance.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (47)
  1. A. Maier, C. Syben, T. Lasser, and C. Riess, “A gentle introduction to deep learning in medical image processing,” Zeitschrift für Medizinische Physik, vol. 29, no. 2, pp. 86–101, 2019.
  2. J. Ker, L. Wang, J. Rao, and T. Lim, “Deep learning applications in medical image analysis,” Ieee Access, vol. 6, pp. 9375–9389, 2017.
  3. S. S. Yadav and S. M. Jadhav, “Deep convolutional neural network based medical image classification for disease diagnosis,” Journal of Big data, vol. 6, no. 1, pp. 1–18, 2019.
  4. S. Azizi, B. Mustafa, F. Ryan, Z. Beaver, J. Freyberg, J. Deaton, A. Loh, A. Karthikesalingam, S. Kornblith, T. Chen et al., “Big self-supervised models advance medical image classification,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 3478–3488.
  5. Y. Dai, Y. Gao, and F. Liu, “Transmed: Transformers advance multi-modal medical image classification,” Diagnostics, vol. 11, no. 8, p. 1384, 2021.
  6. P. Malhotra, S. Gupta, D. Koundal, A. Zaguia, W. Enbeyle et al., “Deep neural networks for medical image segmentation,” Journal of Healthcare Engineering, vol. 2022, 2022.
  7. X.-X. Yin, L. Sun, Y. Fu, R. Lu, Y. Zhang et al., “U-net-based medical image segmentation,” Journal of healthcare engineering, vol. 2022, 2022.
  8. H. Cao, Y. Wang, J. Chen, D. Jiang, X. Zhang, Q. Tian, and M. Wang, “Swin-unet: Unet-like pure transformer for medical image segmentation,” in European conference on computer vision.   Springer, 2022, pp. 205–218.
  9. O. Dalmaz, M. Yurt, and T. Çukur, “Resvit: residual vision transformers for multimodal medical image synthesis,” IEEE Transactions on Medical Imaging, vol. 41, no. 10, pp. 2598–2614, 2022.
  10. T. Zhou, H. Fu, G. Chen, J. Shen, and L. Shao, “Hi-net: hybrid-fusion network for multi-modal mr image synthesis,” IEEE transactions on medical imaging, vol. 39, no. 9, pp. 2772–2781, 2020.
  11. M. Özbey, O. Dalmaz, S. U. Dar, H. A. Bedel, Ş. Özturk, A. Güngör, and T. Çukur, “Unsupervised medical image translation with adversarial diffusion models,” IEEE Transactions on Medical Imaging, 2023.
  12. Z. Chen, Y. Shen, Y. Song, and X. Wan, “Cross-modal memory networks for radiology report generation,” arXiv preprint arXiv:2204.13258, 2022.
  13. S. Yang, X. Wu, S. Ge, Z. Zheng, S. K. Zhou, and L. Xiao, “Radiology report generation with a learned knowledge base and multi-modal alignment,” Medical Image Analysis, vol. 86, p. 102798, 2023.
  14. F. Nooralahzadeh, N. P. Gonzalez, T. Frauenfelder, K. Fujimoto, and M. Krauthammer, “Progressive transformer-based generation of radiology reports,” arXiv preprint arXiv:2102.09777, 2021.
  15. C. Bermudez, A. J. Plassard, L. T. Davis, A. T. Newton, S. M. Resnick, and B. A. Landman, “Learning implicit brain mri manifolds with deep learning,” in Medical imaging 2018: Image processing, vol. 10574, 2018, pp. 408–414.
  16. C. Baur, S. Albarqouni, and N. Navab, “Generating highly realistic images of skin lesions with gans.” arXiv: Computer Vision and Pattern Recognition, Sep 2018.
  17. F. Calimeri, A. Marzullo, C. Stamile, and G. Terracina, “Biomedical data augmentation using generative adversarial neural networks,” in International conference on artificial neural networks, 2017, pp. 626–634.
  18. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” Advances in neural information processing systems, vol. 27, 2014.
  19. J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in neural information processing systems, vol. 33, pp. 6840–6851, 2020.
  20. X. Yi, E. Walia, and P. Babyn, “Generative adversarial network in medical imaging: A review,” Medical image analysis, vol. 58, p. 101552, 2019.
  21. S. Kazeminia, C. Baur, A. Kuijper, B. van Ginneken, N. Navab, S. Albarqouni, and A. Mukhopadhyay, “Gans for medical image analysis,” Artificial Intelligence in Medicine, vol. 109, p. 101938, 2020.
  22. T. Wang, Y. Lei, Y. Fu, W. J. Curran, T. Liu, and X. Yang, “Medical imaging synthesis using deep learning and its clinical applications: A review,” arXiv preprint arXiv:2004.10322, 2020.
  23. M. Arjovsky, S. Chintala, and L. Bottou, “Wasserstein generative adversarial networks,” in International conference on machine learning.   PMLR, 2017, pp. 214–223.
  24. J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli, “Deep unsupervised learning using nonequilibrium thermodynamics,” in International conference on machine learning.   PMLR, 2015, pp. 2256–2265.
  25. P. Dhariwal and A. Nichol, “Diffusion models beat gans on image synthesis,” Advances in neural information processing systems, vol. 34, pp. 8780–8794, 2021.
  26. R. E. Kalman, “A new approach to linear filtering and prediction problems,” 1960.
  27. A. Gu and T. Dao, “Mamba: Linear-time sequence modeling with selective state spaces,” arXiv preprint arXiv:2312.00752, 2023.
  28. Y. Liu, Y. Tian, Y. Zhao, H. Yu, L. Xie, Y. Wang, Q. Ye, and Y. Liu, “Vmamba: Visual state space model,” arXiv preprint arXiv:2401.10166, 2024.
  29. J. Ruan and S. Xiang, “Vm-unet: Vision mamba unet for medical image segmentation,” arXiv preprint arXiv:2402.02491, 2024.
  30. H. Salehinejad, S. Valaee, T. Dowdell, E. Colak, and J. Barfett, “Generalization of deep neural networks for chest pathology classification in x-rays using generative adversarial networks,” in 2018 IEEE international conference on acoustics, speech and signal processing (ICASSP).   IEEE, 2018, pp. 990–994.
  31. C. Han, H. Hayashi, L. Rundo, R. Araki, W. Shimoda, S. Muramatsu, Y. Furukawa, G. Mauri, and H. Nakayama, “Gan-based synthetic brain mr image generation,” in 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), 2018.
  32. H. Zhang, I. Goodfellow, D. Metaxas, and A. Odena, “Self-attention generative adversarial networks,” in International conference on machine learning, 2019.
  33. A. Beers, J. Brown, K. Chang, J. P. Campbell, S. Ostmo, M. F. Chiang, and J. Kalpathy-Cramer, “High-resolution medical image synthesis using progressively grown generative adversarial networks,” arXiv preprint arXiv:1805.03144, 2018.
  34. E. Schonfeld, B. Schiele, and A. Khoreva, “A u-net based discriminator for generative adversarial networks,” in Computer Vision and Pattern Recognition, 2020, pp. 8207–8216.
  35. T. Karras, S. Laine, and T. Aila, “A style-based generator architecture for generative adversarial networks,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 4401–4410.
  36. P. A. Moghadam, S. Van Dalen, K. C. Martin, J. Lennerz, S. Yip, H. Farahani, and A. Bashashati, “A morphology focused diffusion probabilistic model for synthesis of histopathology images,” in Proceedings of the IEEE/CVF winter conference on applications of computer vision, 2023, pp. 2000–2009.
  37. K. Packhäuser, L. Folle, F. Thamm, and A. Maier, “Generation of anonymous chest radiographs using latent diffusion models for training thoracic abnormality classification systems,” in 2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI), pp. 1–5.
  38. M. Aversa, G. Nobis, M. Hägele, K. Standvoss, M. Chirica, R. Murray-Smith, A. M. Alaa, L. Ruff, D. Ivanova, W. Samek et al., “Diffinfinite: Large mask-image synthesis via parallel random patch diffusion in histopathology,” Advances in Neural Information Processing Systems, vol. 36, 2024.
  39. J. Peng, G. Chen, K. Saruta, and Y. Terata, “2d brain mri image synthesis based on lightweight denoising diffusion probabilistic model,” Medical Imaging Process & Technology, vol. 6, no. 1, 2023.
  40. S. Pan, T. Wang, R. L. Qiu, M. Axente, C.-W. Chang, J. Peng, A. B. Patel, J. Shelton, S. A. Patel, J. Roper et al., “2d medical image synthesis using transformer-based denoising diffusion probabilistic model,” Physics in Medicine & Biology, vol. 68, no. 10, p. 105004, 2023.
  41. J. Ma, F. Li, and B. Wang, “U-mamba: Enhancing long-range dependency for biomedical image segmentation,” arXiv preprint arXiv:2401.04722, 2024.
  42. Z. Xing, T. Ye, Y. Yang, G. Liu, and L. Zhu, “Segmamba: Long-range sequential modeling mamba for 3d medical image segmentation,” arXiv preprint arXiv:2401.13560, 2024.
  43. Y. S. Devi and S. P. Kumar, “Dr-dcgan: A deep convolutional generative adversarial network (dc-gan) for diabetic retinopathy image synthesis,” Webology (ISSN: 1735-188X), vol. 19, no. 2, 2022.
  44. J. Song, C. Meng, and S. Ermon, “Denoising diffusion implicit models,” arXiv preprint arXiv:2010.02502, 2020.
  45. C. Sakaridis, D. Dai, and L. Van Gool, “Acdc: The adverse conditions dataset with correspondences for semantic driving scene understanding,” in International Conference on Computer Vision, 2021, pp. 10 765–10 775.
  46. B. H. Menze, A. Jakab, S. Bauer, J. Kalpathy-Cramer, K. Farahani, J. Kirby, Y. Burren, N. Porz, J. Slotboom, R. Wiest et al., “The multimodal brain tumor image segmentation benchmark (brats),” IEEE transactions on medical imaging, vol. 34, no. 10, 2014.
  47. X. Wang, Y. Peng, L. Lu, Z. Lu, M. Bagheri, and R. M. Summers, “Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 2097–2106.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now