Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
184 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Disentangling representations of retinal images with generative models (2402.19186v3)

Published 29 Feb 2024 in cs.CV and cs.LG

Abstract: Retinal fundus images play a crucial role in the early detection of eye diseases. However, the impact of technical factors on these images can pose challenges for reliable AI applications in ophthalmology. For example, large fundus cohorts are often confounded by factors like camera type, bearing the risk of learning shortcuts rather than the causal relationships behind the image generation process. Here, we introduce a population model for retinal fundus images that effectively disentangles patient attributes from camera effects, enabling controllable and highly realistic image generation. To achieve this, we propose a disentanglement loss based on distance correlation. Through qualitative and quantitative analyses, we show that our models encode desired information in disentangled subspaces and enable controllable image generation based on the learned subspaces, demonstrating the effectiveness of our disentanglement loss. The project's code is publicly available: https://github.com/berenslab/disentangling-retinal-images.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (66)
  1. EyePACS digital retinal grading protocol (EyePACS), 2008. URL https://www.eyepacs.org/consultant/Clinical/grading/EyePACS-DIGITAL-RETINAL-IMAGE-GRADING.pdf.
  2. Deep Variational Information Bottleneck. In International Conference on Learning Representations (ICLR), 2017.
  3. ICAM: Interpretable Classification via Disentangled Representations and Feature Attribution Mapping. In Advances in Neural Information Processing Systems, 2020.
  4. ICAM-reg: Interpretable Classification and Regression with Feature Attribution for Mapping Neurological Phenotypes in Individual Scans. In Medical Imaging with Deep Learning, 2021.
  5. Mutual Information Neural Estimation. In Proceedings of the 35th International Conference on Machine Learning, 2018.
  6. Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013.
  7. Measuring Disentanglement: A Review of Metrics. IEEE Transactions on Neural Networks and Learning Systems, 2022.
  8. Causality matters in medical imaging. Nature Communications, 2020.
  9. InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets. In Advances in Neural Information Processing Systems, 2016.
  10. CLUB: A Contrastive Log-ratio Upper Bound of Mutual Information. In Proceedings of the 37th International Conference on Machine Learning, 2020.
  11. A deep learning model for detection of alzheimer’s disease based on retinal photographs: a retrospective, multicentre case-control study. The Lancet Digital Health, 2022.
  12. EyePACS: An Adaptable Telemedicine System for Diabetic Retinopathy Screening. Journal of Diabetes Science and Technology, 2009.
  13. G.A. Darbellay and I. Vajda. Estimation of the information by an adaptive partitioning of the observation space. IEEE Transactions on Information Theory, 1999.
  14. William Falcon and The PyTorch Lightning team. PyTorch Lightning, March 2019.
  15. Avoiding Shortcut-Learning by Mutual Information Minimization in Deep Learning-Based Image Processing. IEEE Access, 2023.
  16. Disentanglement and Generalization Under Correlation Shifts. In ICLR 2022 workshop on Objects, Structure and Causality, 2022.
  17. Unsupervised Domain Adaptation by Backpropagation. In Proceedings of the 32nd International Conference on Machine Learning, 2015.
  18. Domain-Adversarial Training of Neural Networks. Journal of Machine Learning Research, 2016.
  19. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2020.
  20. InvGAN: Invertible GANs. In Pattern Recognition, 2022.
  21. Nonparametric and Semiparametric Models. Springer Berlin, Heidelberg, 2006.
  22. Deep Residual Learning for Image Recognition. In Conference on Computer Vision and Pattern Recognition, 2016.
  23. GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium. In Advances in Neural Information Processing Systems, 2018.
  24. beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework. In International Conference on Learning Representations, 2017.
  25. Towards a Definition of Disentangled Representations, 2018.
  26. Analyzing and Improving the Image Quality of StyleGAN. In Conference on Computer Vision and Pattern Recognition, 2020.
  27. Effects of hypertension, diabetes, and smoking on age and sex prediction from retinal fundus images. Scientific reports, 2020.
  28. Auto-Encoding Variational Bayes. In International Conference on Learning Representations, 2014.
  29. Learning Latent Subspaces in Variational Autoencoders. In Advances in Neural Information Processing Systems, 2018.
  30. Estimating mutual information. Physical Review E, 2004.
  31. Explaining in Style: Training a GAN to explain a classifier in StyleSpace. In International Conference on Computer Vision, 2021.
  32. Diverse Image-to-Image Translation via Disentangled Representations. In European Conference on Computer Vision, 2018.
  33. Learning disentangled representations in the imaging domain. Medical Image Analysis, 2022.
  34. Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations. In Proceedings of the 36th International Conference on Machine Learning, 2019.
  35. The Variational Fair Autoencoder, 2017.
  36. Which Training Methods for GANs do actually Converge? In International Conference on Machine Learning, 2018.
  37. Conditional Generative Adversarial Nets, 2014.
  38. cGANs with Projection Discriminator. In International Conference on Learning Representations, 2018.
  39. Invariant Representations without Adversarial Training. In Advances in Neural Information Processing Systems, 2018.
  40. fundus circle cropping, 2023. URL https://github.com/berenslab/fundus_circle_cropping.
  41. Semi-Supervised StyleGAN for Disentanglement Learning. In Proceedings of the 37th International Conference on Machine Learning, 2020.
  42. Retinal photograph-based deep learning predicts biological age, and stratifies morbidity and mortality risk. Age and ageing, 2022.
  43. Conditional Image Synthesis With Auxiliary Classifier GANs. In Proceedings of the 34th International Conference on Machine Learning, 2017.
  44. Representation Disentanglement for Multi-modal brain MR Analysis. In Information Processing in Medical Imaging, 2021.
  45. On Aliased Resizing and Surprising Subtleties in GAN evaluation. In Conference on Computer Vision and Pattern Recognition, 2022.
  46. PyTorch: An Imperative Style, High-Performance Deep Learning Library, 2019.
  47. opentsne: a modular Python library for t-SNE dimensionality reduction and embedding. 2019.
  48. On Variational Bounds of Mutual Information. In Proceedings of the 36th International Conference on Machine Learning, 2019.
  49. Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning. Nature biomedical engineering, 2018.
  50. PathologyGAN: Learning deep representations of cancer tissue. In Proceedings of the Third Conference on Medical Imaging with Deep Learning, 2021.
  51. Ethnicity is not biology: retinal pigment score to evaluate biological variability from ophthalmic imaging using machine learning, 2023.
  52. Prediction of systemic biomarkers from retinal photographs: development and validation of deep-learning algorithms. The Lancet Digital Health, 2020.
  53. Deep-learning-based cardiovascular risk stratification using coronary artery calcium scores predicted from retinal photographs. The Lancet Digital Health, 2021.
  54. Disentanglement of Correlated Factors via Hausdorff Factorized Support. In International Conference on Learning Representations, 2023.
  55. A deep learning algorithm to detect chronic kidney disease from retinal photographs in community-based populations. The Lancet Digital Health, 2020.
  56. Equivalence of distance-based and RKHS-based statistics in hypothesis testing. The Annals of Statistics, 2013.
  57. Very Deep Convolutional Networks for Large-Scale Image Recognition. 2015.
  58. Predicting high coronary artery calcium score from retinal fundus images with deep learning algorithms. Translational Vision Science & Technology, 2020.
  59. Approximating Mutual Information by Maximum Likelihood Density Ratio Estimation. In Proceedings of the Workshop on New Challenges for Feature Selection in Data Mining and Knowledge Discovery at ECML/PKDD 2008, 2008.
  60. Measuring and testing dependence by correlation of distances. The Annals of Statistics, 2007.
  61. On Disentangled Representations Learned From Correlated Data. In Proceedings of the 38th International Conference on Machine Learning, 2021.
  62. Recent Advances in Autoencoder-Based Representation Learning. In Bayesian Deep Learning Workshop, NeurIPS, 2018.
  63. Validation of a deep-learning-based retinal biomarker (Reti-CVD) in the prediction of cardiovascular disease: data from UK Biobank. BMC medicine, 2023.
  64. T David Williams. Reflections from the Retinal Surface: Some Clinical Implications. Canadian Journal of Optometry, 1982.
  65. Screening and identifying hepatobiliary diseases through deep learning using ocular images: a prospective, multicentre study. The Lancet Digital Health, 2021.
  66. Controllable Invariance through Adversarial Feature Learning. In Advances in Neural Information Processing Systems, 2017.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now