Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Inflating 2D Convolution Weights for Efficient Generation of 3D Medical Images (2208.03934v3)

Published 8 Aug 2022 in eess.IV and cs.CV

Abstract: The generation of three-dimensional (3D) medical images has great application potential since it takes into account the 3D anatomical structure. Two problems prevent effective training of a 3D medical generative model: (1) 3D medical images are expensive to acquire and annotate, resulting in an insufficient number of training images, and (2) a large number of parameters are involved in 3D convolution. Methods: We propose a novel GAN model called 3D Split&Shuffle-GAN. To address the 3D data scarcity issue, we first pre-train a two-dimensional (2D) GAN model using abundant image slices and inflate the 2D convolution weights to improve the initialization of the 3D GAN. Novel 3D network architectures are proposed for both the generator and discriminator of the GAN model to significantly reduce the number of parameters while maintaining the quality of image generation. Several weight inflation strategies and parameter-efficient 3D architectures are investigated. Results: Experiments on both heart (Stanford AIMI Coronary Calcium) and brain (Alzheimer's Disease Neuroimaging Initiative) datasets show that our method leads to improved 3D image generation quality (14.7 improvements on Fr\'echet inception distance) with significantly fewer parameters (only 48.5% of the baseline method). Conclusions: We built a parameter-efficient 3D medical image generation model. Due to the efficiency and effectiveness, it has the potential to generate high-quality 3D brain and heart images for real use cases.

View on arXiv
Definition Search Book Streamline Icon: https://streamlinehq.com
References (55)
  1. Vivit: A video vision transformer, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6836–6846.
  2. Dda-net: Unsupervised cross-modality medical image segmentation via dual domain adaptation. Computer Methods and Programs in Biomedicine 213, 106531.
  3. Learning global dependencies based on hierarchical full connection for brain tumor segmentation. Computer Methods and Programs in Biomedicine 221, 106925.
  4. A short note on the kinetics-700 human action dataset. arXiv preprint arXiv:1907.06987 .
  5. Quo vadis, action recognition? a new model and the kinetics dataset, in: proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6299–6308.
  6. Med3d: Transfer learning for 3d medical image analysis. arXiv preprint arXiv:1904.00625 .
  7. Gan-based generation of realistic 3D data: A systematic review and taxonomy. arXiv preprint arXiv:2207.01390 .
  8. Synthetic data augmentation using GAN for improved liver lesion classification, in: 2018 IEEE 15th international symposium on biomedical imaging (ISBI 2018), IEEE. pp. 289–293.
  9. Transient wall shear stress estimation in coronary bifurcations using convolutional neural networks. Computer Methods and Programs in Biomedicine 225, 107013.
  10. Generative adversarial nets. Advances in neural information processing systems 27.
  11. Coronary calcium score and cardiovascular risk. Journal of the American College of Cardiology 72, 434–447.
  12. Improved training of Wasserstein GANs. Advances in neural information processing systems 30.
  13. Can spatiotemporal 3D CNNs retrace the history of 2D CNNs and Imagenet?, in: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 6546–6555.
  14. Deep residual learning for image recognition, in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778.
  15. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems 30.
  16. 3D-StyleGAN: A style-based generative adversarial network for generative modeling of three-dimensional medical images, in: DGM4MICCAI workshop in International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer. pp. 24–34.
  17. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 .
  18. 3-D brain reconstruction by hierarchical shape-perception network from a single incomplete image. IEEE Transactions on Neural Networks and Learning Systems .
  19. Brain MR to PET synthesis via bidirectional generative adversarial network, in: Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part II 23, Springer. pp. 698–707.
  20. Medical image reconstruction using generative adversarial network for Alzheimer disease assessment with class-imbalance problem, in: 2020 IEEE 6th international conference on computer and communications (ICCC), IEEE. pp. 1323–1327.
  21. Cross-modality synthesis from MRI to PET using adversarial u-net with different normalization, in: 2019 international conference on medical imaging physics and engineering (ICMIPE), IEEE. pp. 1–5.
  22. Squeezenet: Alexnet-level accuracy with 50x fewer parameters and¡ 0.5 mb model size. arXiv preprint arXiv:1602.07360 .
  23. Brain tumor segmentation and radiomics survival prediction: Contribution to the brats 2017 challenge, in: International MICCAI Brainlesion Workshop, Springer. pp. 287–297.
  24. Conditional GAN with an attention-based generator and a 3D discriminator for 3D medical image generation, in: Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part VI 24, Springer. pp. 318–328.
  25. Training generative adversarial networks with limited data. Advances in neural information processing systems 33, 12104–12114.
  26. A style-based generator architecture for generative adversarial networks, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4401–4410.
  27. Analyzing and improving the image quality of stylegan, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 8110–8119.
  28. Attention-aware discrimination for MR-to-CT image translation using cycle-consistent generative adversarial networks. Radiology: Artificial Intelligence 2, e190027.
  29. Compression of deep convolutional neural networks for fast and low power mobile applications. arXiv preprint arXiv:1511.06530 .
  30. Resource efficient 3D convolutional neural networks, in: Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, pp. 0–0.
  31. Factorized higher-order CNNs with an application to spatio-temporal emotion estimation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6060–6069.
  32. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems 25.
  33. Generation of 3D brain MRI using auto-encoding generative adversarial networks, in: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer. pp. 118–126.
  34. Speeding-up convolutional neural networks using fine-tuned cp-decomposition. arXiv preprint arXiv:1412.6553 .
  35. A review of cardiac image registration methods. IEEE Transactions on medical imaging 21, 1011–1021.
  36. Moments in time dataset: one million videos for event understanding. IEEE transactions on pattern analysis and machine intelligence 42, 502–508.
  37. Circle representation for medical object detection. IEEE Transactions on Medical Imaging 41, 746–754.
  38. Tensorizing neural networks. Advances in neural information processing systems 28.
  39. Compressed sensing MRI reconstruction using a generative adversarial network with a cyclic loss. IEEE transactions on medical imaging 37, 1488–1497.
  40. Degenerative adversarial neuroimage nets for brain scan simulations: Application in ageing and dementia. Medical Image Analysis 75, 102257.
  41. Variational approaches for auto-encoding generative adversarial networks. arXiv preprint arXiv:1706.04987 .
  42. Imagenet large scale visual recognition challenge. International Journal of Computer Vision 115, 211–252.
  43. 3D convolutional neural networks for stalled brain capillary detection. Computers in Biology and Medicine 141, 105089.
  44. Deep learning-based automatic segmentation of images in cardiac radiography: a promising challenge. Computer Methods and Programs in Biomedicine , 106821.
  45. Stanford, 2022. Coca - coronary calcium and chest ct’s dataset. URL: https://stanfordaimi.azurewebsites.net/datasets/e8ca74dc-8dd4-4340-815a-60b41f6cb2aa.
  46. Rethinking the inception architecture for computer vision, in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2818–2826.
  47. Minegan: effective knowledge transfer from GANs to target domains with few images, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9332–9341.
  48. The Alzheimer’s disease neuroimaging initiative 3: Continued innovation for clinical trial improvement. Alzheimer’s & Dementia 13, 561–571.
  49. A survey of transfer learning. Journal of Big Data 3, 1–40.
  50. Learning a probabilistic latent space of object shapes via 3D generative-adversarial modeling. Advances in neural information processing systems 29.
  51. Deep learning based MRI reconstruction with transformer. Computer Methods and Programs in Biomedicine 233, 107452.
  52. Low-dose CT image denoising using a generative adversarial network with Wasserstein distance and perceptual loss. IEEE transactions on medical imaging 37, 1348–1357.
  53. Fine perceptive GANs for brain MR image super-resolution in wavelet domain. IEEE transactions on neural networks and learning systems .
  54. Shufflenet: An extremely efficient convolutional neural network for mobile devices, in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 6848–6856.
  55. 3D segmentation guided style-based generative adversarial networks for PET synthesis. IEEE Transactions on Medical Imaging 41, 2092–2104.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Yanbin Liu (18 papers)
  2. Girish Dwivedi (10 papers)
  3. Farid Boussaid (30 papers)
  4. Frank Sanfilippo (3 papers)
  5. Makoto Yamada (84 papers)
  6. Mohammed Bennamoun (124 papers)
Citations (7)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now