3DGAUnet: 3D generative adversarial networks with a 3D U-Net based generator to achieve the accurate and effective synthesis of clinical tumor image data for pancreatic cancer (2311.05697v2)
Abstract: Pancreatic ductal adenocarcinoma (PDAC) presents a critical global health challenge, and early detection is crucial for improving the 5-year survival rate. Recent medical imaging and computational algorithm advances offer potential solutions for early diagnosis. Deep learning, particularly in the form of convolutional neural networks (CNNs), has demonstrated success in medical image analysis tasks, including classification and segmentation. However, the limited availability of clinical data for training purposes continues to provide a significant obstacle. Data augmentation, generative adversarial networks (GANs), and cross-validation are potential techniques to address this limitation and improve model performance, but effective solutions are still rare for 3D PDAC, where contrast is especially poor owing to the high heterogeneity in both tumor and background tissues. In this study, we developed a new GAN-based model, named 3DGAUnet, for generating realistic 3D CT images of PDAC tumors and pancreatic tissue, which can generate the interslice connection data that the existing 2D CT image synthesis models lack. Our innovation is to develop a 3D U-Net architecture for the generator to improve shape and texture learning for PDAC tumors and pancreatic tissue. Our approach offers a promising path to tackle the urgent requirement for creative and synergistic methods to combat PDAC. The development of this GAN-based model has the potential to alleviate data scarcity issues, elevate the quality of synthesized data, and thereby facilitate the progression of deep learning models to enhance the accuracy and early detection of PDAC tumors, which could profoundly impact patient outcomes. Furthermore, this model has the potential to be adapted to other types of solid tumors, hence making significant contributions to the field of medical imaging in terms of image processing models.
- Society, A.C. Cancer Facts & Figures 2022. Technical report, American Cancer Society, 2022.
- Program, S.R. Surveillance, Epidemiology, and End Results (SEER) Program (www.seer.cancer.gov) SEER*Stat Database: Populations - Total U.S. (1969-2020) [Katrina/Rita Adjustment] - Linked To County Attributes - Total U.S., 1969-2020 Counties. Technical report, National Cancer Institute, DCCPS, 2022.
- A survey of pulmonary nodule detection, segmentation and classification in computed tomography with deep learning techniques. Journal of Medical Artificial Intelligence 2019, 2.
- A review on a deep learning perspective in brain cancer classification. Cancers 2019, 11, 111.
- A comprehensive review on multi-organs tumor detection based on machine learning. Pattern Recognition Letters 2020, 131, 30–37.
- Performance and clinical applicability of machine learning in liver computed tomography imaging: a systematic review. European Radiology 2023, pp. 1–29.
- Comparison of deep learning approaches for multi-label chest X-ray classification. Scientific reports 2019, 9, 6381.
- Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning. Nature medicine 2018, 24, 1559–1567.
- Deep learning for remote sensing image classification: A survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 2018, 8, e1264.
- Deep learning for medical image processing: Overview, challenges and the future. Classification in BioApps: Automation of Decision Making 2018, pp. 323–350.
- Deep learning for hyperspectral image classification: An overview. IEEE Transactions on Geoscience and Remote Sensing 2019, 57, 6690–6709.
- Pancreatic cancer imaging: a new look at an old problem. Current Problems in Diagnostic Radiology 2021, 50, 540–550.
- Fully end-to-end deep-learning-based diagnosis of pancreatic tumors. Theranostics 2021, 11, 1982.
- Sharpness-aware minimization for efficiently improving generalization. arXiv preprint arXiv:2010.01412 2020.
- Pancreatic Image Augmentation Based on Local Region Texture Synthesis for Tumor Segmentation. In Proceedings of the International Conference on Artificial Neural Networks. Springer, 2022, pp. 419–431.
- Medical image augmentation for lesion detection using a texture-constrained multichannel progressive GAN. Computers in Biology and Medicine 2022, 145, 105444.
- Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling. Advances in neural information processing systems 2016, 29.
- The medical segmentation decathlon. Nature communications 2022, 13, 4128.
- Hounsfield, G.N. Computed medical imaging. Science 1980, 210, 22–28.
- Attention u-net: Learning where to look for the pancreas. arXiv preprint arXiv:1804.03999 2018.
- Deep Image Blending. In Proceedings of the The IEEE Winter Conference on Applications of Computer Vision, 2020, pp. 231–240.
- 3D Slicer: a platform for subject-specific image analysis, visualization, and clinical support. In Intraoperative imaging and image-guided therapy; Springer, 2013; pp. 277–289.
- Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems 2017, 30.
- Rethinking the inception architecture for computer vision. In Proceedings of the Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2818–2826.
- A kernel two-sample test. The Journal of Machine Learning Research 2012, 13, 723–773.
- Conditional image synthesis with auxiliary classifier gans. In Proceedings of the International conference on machine learning. PMLR, 2017, pp. 2642–2651.
- Learning spatiotemporal features with 3d convolutional networks. In Proceedings of the Proceedings of the IEEE international conference on computer vision, 2015, pp. 4489–4497.