Echocardiography video synthesis from end diastolic semantic map via diffusion model (2310.07131v1)
Abstract: Denoising Diffusion Probabilistic Models (DDPMs) have demonstrated significant achievements in various image and video generation tasks, including the domain of medical imaging. However, generating echocardiography videos based on semantic anatomical information remains an unexplored area of research. This is mostly due to the constraints imposed by the currently available datasets, which lack sufficient scale and comprehensive frame-wise annotations for every cardiac cycle. This paper aims to tackle the aforementioned challenges by expanding upon existing video diffusion models for the purpose of cardiac video synthesis. More specifically, our focus lies in generating video using semantic maps of the initial frame during the cardiac cycle, commonly referred to as end diastole. To further improve the synthesis process, we integrate spatial adaptive normalization into multiscale feature maps. This enables the inclusion of semantic guidance during synthesis, resulting in enhanced realism and coherence of the resultant video sequences. Experiments are conducted on the CAMUS dataset, which is a highly used dataset in the field of echocardiography. Our model exhibits better performance compared to the standard diffusion technique in terms of multiple metrics, including FID, FVD, and SSMI.
- “Artificial intelligence in echocardiography: detection, functional evaluation, and disease diagnosis,” Cardiovascular ultrasound, vol. 19, no. 1, pp. 1–11, 2021.
- “Deep learning interpretation of echocardiograms,” NPJ digital medicine, vol. 3, no. 1, pp. 10, 2020.
- G. Varoquaux and V. Cheplygina, “Machine learning for medical imaging: methodological failures and recommendations for the future,” NPJ Digital Medicine, vol. 5, no. 1, pp. 48, 2022.
- “Generating synthetic labeled data from existing anatomical models: An example with echocardiography segmentation,” IEEE Trans Med Imaging, vol. 40, pp. 2783–2794, 10 2021.
- “Real-Time GPU-Based Ultrasound Simulation Using Deformable Mesh Models,” IEEE Trans Med Imaging, vol. 32, pp. 609–618, March 2013.
- D. Garcia, “Simus: an open-source simulator for medical ultrasound imaging. part i: theory & examples,” Computer Methods and Programs in Biomedicine, vol. 218, pp. 106726, 2022.
- “Feature-conditioned cascaded video diffusion models for precise echocardiogram synthesis,” 2023.
- “Echo from noise: synthetic ultrasound image generation using diffusion models for real image segmentation,” arXiv preprint arXiv:2305.05424, 2023.
- “Weakly-supervised high-fidelity ultrasound video synthesis with feature decoupling,” in Proc. MICCAI, LNCS. Springer, 2022, pp. 310–319.
- “Sketch guided and progressive growing GAN for realistic and editable ultrasound image synthesis,” Medical Image Analysis, vol. 79, pp. 102461, July 2022.
- “Bias and generalization in deep generative models: An empirical study,” Proc. NeurIPS, vol. 31, 2018.
- “Denoising Diffusion Probabilistic Models,” in Proc. NeurIPS, 2020, vol. 33, pp. 6840–6851.
- “Video-to-video synthesis,” arXiv preprint arXiv:1808.06601, 2018.
- “Video generation from single semantic label map,” in Proc. CVPR, 2019, pp. 3733–3742.
- “Video Diffusion Models,” June 2022, arXiv:2204.03458.
- “Palette: Image-to-image diffusion models,” 2022.
- “Semantic image synthesis via diffusion models,” 2022.
- J. Ho and T. Salimans, “Classifier-Free Diffusion Guidance,” July 2022, arXiv:2207.12598.
- P. Dhariwal and A. Nichol, “Diffusion models beat gans on image synthesis,” 2021.
- “Deep learning for segmentation using an open large-scale dataset in 2d echocardiography,” IEEE Trans Med Imaging, vol. 38, no. 9, pp. 2198–2210, 2019.
- “Cascaded Diffusion Models for High Fidelity Image Generation,” Journal of Machine Learning Research, vol. 23, pp. 1–33, 2022.
- “Gans trained by a two time-scale update rule converge to a local nash equilibrium,” Neurips, vol. 30, 2017.
- “Towards accurate generative models of video: A new metric & challenges,” arXiv preprint arXiv:1812.01717, 2018.
- “Image quality assessment: from error visibility to structural similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600–612, 2004.