OCT2Confocal: 3D CycleGAN based Translation of Retinal OCT Images to Confocal Microscopy (2311.10902v3)
Abstract: Optical coherence tomography (OCT) and confocal microscopy are pivotal in retinal imaging, each presenting unique benefits and limitations. In-vivo OCT offers rapid, non-invasive imaging but can be hampered by clarity issues and motion artifacts. Ex-vivo confocal microscopy provides high-resolution, cellular detailed color images but is invasive and poses ethical concerns and potential tissue damage. To bridge these modalities, we developed a 3D CycleGAN framework for unsupervised translation of in-vivo OCT to ex-vivo confocal microscopy images. Applied to our OCT2Confocal dataset, this framework effectively translates between 3D medical data domains, capturing vascular, textural, and cellular details with precision. This marks the first attempt to exploit the inherent 3D information of OCT and translate it into the rich, detailed color domain of confocal microscopy. Assessed through quantitative and qualitative evaluations, the 3D CycleGAN framework demonstrates commendable image fidelity and quality, outperforming existing methods despite the constraints of limited data. This non-invasive generation of retinal confocal images has the potential to further enhance diagnostic and monitoring capabilities in ophthalmology. Our source code and OCT2Confocal dataset are available at https://github.com/xintian-99/OCT2Confocal.
- “Retinal imaging and image analysis,” IEEE Reviews in Biomedical Engineering, vol. 3, pp. 169–208, 2010.
- “Microstructure and network organization of the microvasculature in the human macula,” Investigative ophthalmology & visual science, vol. 51, no. 12, pp. 6735–6743, 2010.
- “Generative adversarial nets,” in Advances in Neural Information Processing Systems (NeurIPS). 2014, vol. 27, Curran Associates, Inc.
- “Translation of color fundus photography into fluorescein angiography using deep learning for enhanced diabetic retinopathy screening,” Ophthalmology Science, vol. 3, no. 4, pp. 100401, 2023.
- “Image-to-image translation with conditional adversarial networks,” in Proceedings of the IEEE conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 1125–1134.
- “Unpaired image-to-image translation using cycle-consistent adversarial networks,” in 2017 IEEE International Conference on Computer Vision (ICCV), 2017, pp. 2242–2251.
- “Deep learning methods to generate synthetic ct from mri in radiotherapy: A literature review,” Physica Medica, vol. 89, pp. 265–281, 2021.
- “U-net: Convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18. Springer, 2015, pp. 234–241.
- “Wasserstein generative adversarial networks,” in International conference on machine learning (ICML). PMLR, 2017, pp. 214–223.
- “Imaging study of pseudo-CT synthesized from cone-beam CT based on 3D CycleGAN in radiotherapy,” Frontiers in oncology, vol. 11, pp. 603844, 2021.
- “GANs trained by a two time-scale update rule converge to a local Nash equilibrium,” in Advances in Neural Information Processing Systems (NeurIPS). 2017, vol. 30, Curran Associates, Inc.
- “Demystifying MMD GANs,” arXiv preprint arXiv:1801.01401, 2018.
- “Unpaired image-to-image translation via neural Schrödinger bridge,” arXiv preprint arXiv:2305.15086, 2023.