Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
129 tokens/sec
GPT-4o
28 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

DISCOVER: 2-D Multiview Summarization of Optical Coherence Tomography Angiography for Automatic Diabetic Retinopathy Diagnosis (2401.05137v1)

Published 10 Jan 2024 in eess.IV and cs.CV

Abstract: Diabetic Retinopathy (DR), an ocular complication of diabetes, is a leading cause of blindness worldwide. Traditionally, DR is monitored using Color Fundus Photography (CFP), a widespread 2-D imaging modality. However, DR classifications based on CFP have poor predictive power, resulting in suboptimal DR management. Optical Coherence Tomography Angiography (OCTA) is a recent 3-D imaging modality offering enhanced structural and functional information (blood flow) with a wider field of view. This paper investigates automatic DR severity assessment using 3-D OCTA. A straightforward solution to this task is a 3-D neural network classifier. However, 3-D architectures have numerous parameters and typically require many training samples. A lighter solution consists in using 2-D neural network classifiers processing 2-D en-face (or frontal) projections and/or 2-D cross-sectional slices. Such an approach mimics the way ophthalmologists analyze OCTA acquisitions: 1) en-face flow maps are often used to detect avascular zones and neovascularization, and 2) cross-sectional slices are commonly analyzed to detect macular edemas, for instance. However, arbitrary data reduction or selection might result in information loss. Two complementary strategies are thus proposed to optimally summarize OCTA volumes with 2-D images: 1) a parametric en-face projection optimized through deep learning and 2) a cross-sectional slice selection process controlled through gradient-based attribution. The full summarization and DR classification pipeline is trained from end to end. The automatic 2-D summary can be displayed in a viewer or printed in a report to support the decision. We show that the proposed 2-D summarization and classification pipeline outperforms direct 3-D classification with the advantage of improved interpretability.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (69)
  1. Multiclass Classification Performance Curve. IEEE Access 10, 68915–68921. doi:10.1109/ACCESS.2022.3186444. conference Name: IEEE Access.
  2. Supervised machine learning based multi-task artificial intelligence classification of retinopathies. J Clin Med 8, 872. doi:10.3390/jcm8060872.
  3. DR detection using optical coherence tomography angiography (OCTA): A transfer learning approach with robustness analysis, in: Proc MICCAI OMIA Works, Lima, Peru. pp. 11–20. doi:10.1007/978-3-030-63419-3_2.
  4. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS One 10, e0130140. doi:10.1371/journal.pone.0130140.
  5. Total retinal blood flow measurement with ultrahigh speed swept source/Fourier domain OCT. Biomed Opt Express 2, 1539–1552. doi:10.1364/BOE.2.001539.
  6. Radiomics-based assessment of OCT angiography images for diabetic retinopathy diagnosis. Ophthalmol Sci 3, 100259. doi:10.1016/j.xops.2022.100259.
  7. The Airlie classification of diabetic retinopathy, in: Goldberg, M., Fine, S. (Eds.), Proc Symp Treatment of Diabetic Retinopathy, Washington, DC, USA. p. 1890.
  8. ETDRS Research Group, 1991. Fundus photographic risk factors for progression of diabetic retinopathy: ETDRS report number 12. Ophthalmology 98, 823–833. doi:10.1016/S0161-6420(13)38014-2.
  9. Automatic registration of CT volumes and dual-energy digital radiography for detection of cardiac and lung diseases, in: Proc IEEE EMBC, New York, NY, USA. pp. 1976–1979. doi:10.1109/IEMBS.2006.259888.
  10. High-throughput widefield fluorescence imaging of 3D samples using deep learning for 2D projection image restoration. PLoS ONE 17, e0264241. doi:10.1371/journal.pone.0264241.
  11. Comparison of amplitude-decorrelation, speckle-variance and phase-variance OCT angiography methods for imaging the human retina and choroid. Biomed Opt Express 7, 911–942. doi:10.1364/BOE.7.000911.
  12. Automatic quantification of superficial foveal avascular zone in optical coherence tomography angiography implemented with deep learning. Vis Comput Ind Biomed Art 2, 21. doi:10.1186/s42492-019-0031-8.
  13. Automated segmentation of retinal fluid volumes from structural and angiographic optical coherence tomography using deep learning. Transl Vis Sci Technol 9, 54. doi:10.1167/tvst.9.2.54.
  14. An end-to-end network for segmenting the vasculature of three retinal capillary plexuses from OCT angiographic volumes. Biomed Opt Express 12, 4889–4900. doi:10.1364/BOE.431888. publisher: Optica Publishing Group.
  15. Performance of a deep neural network algorithm based on a small medical image dataset: Incremental impact of 3D-to-2D reformation combined with novel data augmentation, photometric conversion, or transfer learning. J Digit Imaging 33, 431–438. doi:10.1007/s10278-019-00267-3.
  16. DeepProjection: Specific and robust projection of curved 2D tissue sheets from 3D microscopy using deep learning. Dev (Camb) 149, dev200621. doi:10.1242/dev.200621.
  17. Can spatiotemporal 3D CNNs retrace the history of 2D CNNs and ImageNet?, in: Proc CVPR, IEEE Computer Society, Los Alamitos, CA, USA. pp. 6546–6555. doi:10.1109/CVPR.2018.00685.
  18. Deep residual learning for image recognition, in: Proc CVPR, Las Vegas, NV, USA. pp. 770–778. doi:10.1109/CVPR.2016.90.
  19. Ensemble deep learning for diabetic retinopathy detection using optical coherence tomography angiography. Transl Vis Sci Technol 9, 20. doi:10.1167/tvst.9.2.20.
  20. Three-dimensional imaging of the human retina by high-speed optical coherence tomography. Opt Express 11, 2753–2761. doi:10.1364/OE.11.002753.
  21. Squeeze-and-excitation networks. IEEE Trans Pattern Anal Mach Intell 42, 2011–2023. URL: https://doi.org/10.1109/TPAMI.2019.2913372, doi:10.1109/TPAMI.2019.2913372.
  22. Batch normalization: Accelerating deep network training by reducing internal covariate shift, in: Proc ICML, JMLR.org, Lille, France. pp. 448–456.
  23. Self-supervised 2D/3D registration for X-ray to CT image fusion, in: Proc IEEE/CVF WACV, Waikoloa, HI, USA. pp. 2787–2797. doi:10.1109/WACV56688.2023.00281.
  24. Automated machine learning–based classification of proliferative and non-proliferative diabetic retinopathy using optical coherence tomography angiography vascular density maps. Graefes Arch Clin Exp Ophthalmol 261, 391–399. doi:10.1007/s00417-022-05818-z.
  25. ImageNet classification with deep convolutional neural networks, in: Proc NIPS, Granada, Spain. pp. 1097–1105.
  26. Projective skip-connections for segmentation along a subset of dimensions in retinal OCT, in: de Bruijne, M., Cattin, P.C., Cotin, S., Padoy, N., Speidel, S., Zheng, Y., Essert, C. (Eds.), Proc MICCAI, Springer International Publishing, Strasbourg, France. pp. 431–441. doi:10.1007/978-3-030-87193-2_41.
  27. Novel application of long short-term memory network for 3D to 2D retinal vessel segmentation in adaptive optics— optical coherence tomography volumes. Appl Sci 11. doi:10.3390/app11209475.
  28. Transfer learning for automated OCTA detection of diabetic retinopathy. Transl Vis Sci Technol 9, 35. doi:10.1167/tvst.9.2.35.
  29. Image projection network: 3D to 2D image segmentation in OCTA images. IEEE Trans Med Imaging 39, 3343–3354. doi:10.1109/TMI.2020.2992244.
  30. Diagnosing diabetic retinopathy in OCTA images based on multilevel information fusion using a deep learning framework. Comput Math Methods Med 2022, 4316507. doi:10.1155/2022/4316507.
  31. Multimodal information fusion for glaucoma and diabetic retinopathy classification, in: Proc MICCAI OMIA Works, Singapore, Singapore. pp. 53–62. doi:10.1007/978-3-031-16525-2_6.
  32. 3-D analysis of multiple OCTA acquisitions for the automatic diagnosis of diabetic retinopathy, in: Proc ARVO, New Orleans, LA, USA. p. C0116.
  33. A ConvNet for the 2020s, in: Proc CVPR, IEEE Computer Society, Los Alamitos, CA, USA. pp. 11966–11976. doi:10.1109/CVPR52688.2022.01167.
  34. Federated learning for microvasculature segmentation and diabetic retinopathy classification of OCT data. Ophthal Sci 1, 100069. doi:10.1016/j.xops.2021.100069.
  35. Computer-aided diagnosis of melanoma subtypes using reflectance confocal images. Cancers 15, 1428. doi:10.3390/cancers15051428.
  36. Ultra-widefield OCT angiography. IEEE Trans Med Imaging 42, 1009–1020. doi:10.1109/TMI.2022.3222638.
  37. IDF Diabetes Atlas: Global estimates for the prevalence of diabetes for 2015 and 2040. Diabetes Res Clin Pract 128, 40–50. doi:10.1016/j.diabres.2017.03.024.
  38. ExplAIn: Explanatory artificial intelligence for diabetic retinopathy diagnosis. Med Image Anal 72, 102118. doi:10.1016/j.media.2021.102118.
  39. Deep image mining for diabetic retinopathy screening. Med Image Anal 39, 178–193. doi:10.1016/j.media.2017.04.012.
  40. ”why should I trust you?”: Explaining the predictions of any classifier, in: Proc KDD, Association for Computing Machinery, New York, NY, USA. pp. 1135–1144. doi:10.1145/2939672.2939778.
  41. U-Net: Convolutional networks for biomedical image segmentation, in: Proc MICCAI, Munich, Germany. pp. 234–241. doi:10.1007/978-3-319-24574-4_28.
  42. A deep learning algorithm for classifying diabetic retinopathy using optical coherence tomography angiography. Transl Vis Sci Technol 11, 39. doi:10.1167/tvst.11.2.39.
  43. A deep learning model for identifying diabetic retinopathy using optical coherence tomography angiography. Sci Rep 11, 23024. doi:10.1038/s41598-021-02479-6.
  44. Evaluating the visualization of what a deep neural network has learned. IEEE Trans Neural Netw Learn Syst 28, 2660–2673. doi:10.1109/TNNLS.2016.2599820.
  45. Metric-driven learning of correspondence weighting for 2-D/3-D image registration, in: Proc GCPR, Stuttgart, Germany. pp. 140–152. doi:10.1007/978-3-030-12939-2_11.
  46. Efficient 3D junction detection in biomedical images based on a circular sampling model and reverse mapping. IEEE J Biomed Health Inform 25, 1612–1623. doi:10.1109/JBHI.2020.3036743.
  47. Learning important features through propagating activation differences, in: Proc ICML, JMLR.org, Sydney, NSW, Australia. pp. 3145–3153.
  48. Deep inside convolutional networks: Visualising image classification models and saliency maps, in: Proc ICLR Works, Calgary, Canada.
  49. Striving for simplicity: The all convolutional net, in: Bengio, Y., LeCun, Y. (Eds.), Proc ICLR Works, San Diego, CA, USA. URL: http://arxiv.org/abs/1412.6806.
  50. Deep learning-based automatic assessment of lung impairment in COVID-19 pneumonia: Predicting markers of hypoxia with computer vision. Front Med 9, 882190. doi:10.3389/fmed.2022.882190.
  51. Optical coherence tomography angiography in diabetic retinopathy: An updated review. Eye 35, 149–161. doi:10.1038/s41433-020-01233-y.
  52. Axiomatic attribution for deep networks, in: Proc ICML, JMLR.org, Sydney, NSW, Australia. pp. 3319–3328.
  53. EfficientNet: Rethinking model scaling for convolutional neural networks, in: Proc ICML, Long Beach, CA, USA.
  54. EfficientNetV2: Smaller models and faster training, in: Proc ICML, Virtual. pp. 10096–10106. URL: http://arxiv.org/abs/2104.00298.
  55. The impact of averaging logits over probabilities on ensembles of neural networks, in: Proc Works AISafety, Vienna, Austria.
  56. Deep learning in ophthalmology: The technical and clinical considerations. Prog Retin Eye Res 72, 100759. doi:https://doi.org/10.1016/j.preteyeres.2019.04.003.
  57. Deep learning-based 2D/3D registration of an atlas to biplanar X-ray images. Int J Comput Assist Radiol Surg 17, 1333–1342. doi:10.1007/s11548-022-02586-3.
  58. Update on optical coherence tomography and optical coherence tomography angiography imaging in proliferative diabetic retinopathy. Diagnostics 11, 1869. doi:10.3390/diagnostics11101869.
  59. Standardization of optical coherence tomography angiography imaging biomarkers in diabetic retinal disease. Ophthalmic Res 64, 871–887. doi:10.1159/000518620.
  60. VC-Net: Deep volume-composition networks for segmentation and visualization of highly sparse and noisy image data. IEEE Trans Comput Imaging 27, 1301–1311. doi:10.1109/TVCG.2020.3030374.
  61. Proposed international clinical diabetic retinopathy and diabetic macular edema disease severity scales. Ophthalmology 110, 1677–1682. doi:10.1016/S0161-6420(03)00475-5.
  62. AV-casnet: Fully automatic arteriole-venule segmentation and differentiation in OCT angiography. IEEE Trans Med Imaging 42, 481–492. doi:10.1109/TMI.2022.3214291.
  63. Deep learning in optical coherence tomography angiography: Current progress, challenges, and future directions. Diagnostics 13, 326. doi:10.3390/diagnostics13020326.
  64. Classification of diabetic retinopathy: Past, present and future. Front Endocrinol 13, 1079217. doi:10.3389/fendo.2022.1079217.
  65. Automated diagnosis of optical coherence tomography angiography (OCTA) based on machine learning techniques. Sensors 22, 2342. doi:10.3390/s22062342.
  66. Global prevalence and major risk factors of diabetic retinopathy. Diabetes Care 35, 556–564. doi:10.2337/dc11-1909.
  67. Deep-learning–aided diagnosis of diabetic retinopathy, age-related macular degeneration, and glaucoma based on structural and angiographic OCT. Ophthalmol Sci 3, 1000245. doi:10.1016/j.xops.2022.100245.
  68. A diabetic retinopathy classification framework based on deep-learning analysis of OCT angiography. Transl Vis Sci Technol 11, 10. doi:10.1167/tvst.11.7.10.
  69. Visualizing and understanding convolutional networks, in: Proc ECCV, Zurich, Switzerland. pp. 818–833.
Citations (7)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now