RETFound: Retinal Imaging Foundation Model
- RETFound is a retinal imaging foundation model that leverages self-supervised masked autoencoding and Vision Transformer architecture on large-scale unlabeled retinal images.
- It employs a multi-stage pretraining process—first on ImageNet then on retinal datasets—to effectively support clinical tasks like classification, segmentation, and multimodal fusion.
- RETFound has become a key reference in ophthalmology, driving advances in systemic disease prediction, fairness assessments, and deployment in resource-constrained clinical settings.
Searching arXiv for RETFound and closely related retinal foundation-model papers to ground the article. RETFound is a retinal imaging foundation model for color fundus photography and optical coherence tomography that uses a vision transformer and masked autoencoding to learn transferable representations from large-scale unlabeled retinal images, then adapts those representations to downstream clinical tasks with explicit labels. Review literature describes it as the first foundation model for ophthalmology and characterizes a canonical RETFound configuration as a vision model for CFP and OCT built on ViT-Large, pretrained with MAE on 1.6 million retinal images, with 307 million parameters (Veldhuizen et al., 10 Jun 2025). Subsequent work has treated RETFound both as a high-performing retinal baseline and as a reusable backbone for classification, segmentation, multimodal fusion, uncertainty-aware diagnosis, and oculomics, while also testing when retina-specific pretraining justifies its scale and computational cost (Peng et al., 2024, Zhao et al., 15 Aug 2025, Isztl et al., 27 Nov 2025).
1. Historical position and model identity
RETFound occupies a foundational position in the retinal foundation-model literature. A broad review of medical imaging foundation models identifies it as the first foundation model for ophthalmology and places it at the start of a line of vision-only retinal models that learn from large-scale unlabeled retinal images rather than from image–report pairs (Veldhuizen et al., 10 Jun 2025). In that review, RETFound is explicitly listed as a vision model for CFP and OCT, with a ViT-Large backbone and MAE pretraining, and is presented as a reusable encoder for downstream ophthalmic and systemic prediction tasks (Veldhuizen et al., 10 Jun 2025).
Task-specific papers give more granular but partially different summaries of the original pretraining pipeline. One head-to-head comparison against DINOv2 describes RETFound as a Vision Transformer trained sequentially on 1.4 million natural images and then on roughly 1.6 million retinal images, including about 900,000 color fundus photographs and about 730,000 OCT scans (Hou et al., 10 Feb 2025). Another comparative study states that RETFound was pretrained sequentially with MAE on ImageNet-1k, 904,170 unlabeled CFPs, and 736,442 OCT scans from MEH-MIDAS and EyePACS (Yew et al., 21 Jan 2025). A fairness study focused on BRSET likewise describes RETFound as MAE-pretrained, first on ImageNet-1k and then on 904,170 CFPs, with 90.2% from MEH-MIDAS and 9.8% from EyePACS (Queiroz et al., 2024). Taken together, these papers agree on the core identity of RETFound as a retina-specific, self-supervised, ViT-based foundation model, even when they differ in how much of the original multi-stage training recipe they restate.
2. Architecture, pretraining paradigm, and model variants
RETFound is consistently described as a ViT-family model trained with masked autoencoding. A segmentation adaptation paper gives the clearest architectural summary used in downstream work: a ViT-Large encoder with 24 transformer blocks, embedding dimension 1024, 16 attention heads, and patch size (Zhao et al., 15 Aug 2025). In downstream adaptation studies, RETFound commonly appears as an encoder whose original MAE decoder is discarded and replaced with task-specific heads, which is consistent with the MAE formulation summarized in the review literature (Veldhuizen et al., 10 Jun 2025, Peng et al., 2024).
The literature does not treat RETFound as a single immutable checkpoint. Instead, several named variants appear. A systematic benchmarking paper evaluates RETFound-MAE-OCT, RETFound-MAE-CFP, and RETFound-DinoV2-CFP, all at 303 million parameters, to test when domain-specific retinal pretraining is warranted across OCT, diabetic macular edema, diabetic retinopathy, and glaucoma tasks (Isztl et al., 27 Nov 2025). A specialist-versus-generalist study compares RETFound-MAE and RETFound-DINOv2, both using ViT-large backbones and both pretrained on 904K retinal images from AlzEye, and finds RETFound-DINOv2 to be the stronger specialist model in that benchmark (Zhou et al., 3 Sep 2025). This family structure is important: “RETFound” names both the original retinal foundation-model idea and a continuing line of retinally specialized ViT checkpoints.
At the same time, many downstream papers do not restate the original pretraining objective in full mathematical detail. Several explicitly note that the original masking strategy, optimizer, full data composition, and fine-tuning hyperparameters must be obtained from the original RETFound paper or reference implementation rather than from the downstream application paper itself (Peng et al., 2024, Turkan et al., 7 Nov 2025). This has made RETFound a widely used reference model, but not always a fully re-specified one.
3. Downstream adaptation patterns
A striking feature of the RETFound literature is the variety of adaptation regimes built on top of the pretrained encoder. The simplest pattern is standard supervised transfer, either by full fine-tuning or by linear probing. A DME benchmark compares RETFound under exactly those two regimes and reports that full fine-tuning is consistently better than linear probing for that task, implying that RETFound is not especially effective there as a frozen off-the-shelf feature extractor (Arellano et al., 8 Oct 2025). A multimodal hypertension study similarly reports that both linear probing and fine-tuning were evaluated, with fine-tuning producing higher validation AUC for RETFound (Baharoon et al., 2023).
Other studies use RETFound as a frozen or partially frozen backbone inside more elaborate systems. In “Enhancing Diagnostic Reliability of Foundation Model with Uncertainty Estimation in OCT Images” (Peng et al., 2024), the authors discard the decoder of RETFound, keep its encoder as backbone, freeze the image encoder, insert trainable LoRA layers, and add an uncertainty-based classifier. In HyMNet, RETFound forms the fundus branch of a multimodal network and outputs an 8-dimensional feature vector that is concatenated with a 32-dimensional demographic representation derived from age and gender (Baharoon et al., 2023). In a cascaded rural screening architecture, RETFound-DINOv2 serves as the cloud-side specialist model: a ViT-L/14 backbone with a newly initialized head, partial fine-tuning of the last two transformer blocks and fc_norm, and an ordinal-regression formulation for diabetic retinopathy grading (Doshi et al., 13 May 2026).
RETFound has also been adapted beyond image-level classification. TAPE applies RETFound to multimodal OCT/OCTA retinal layer segmentation through a two-stage parameter-efficient pipeline: stage 1 performs domain alignment by masked image modeling of OCTA only, and stage 2 performs task fitting via a second LoRA adapter plus a convolutional segmentation head (Su et al., 6 Apr 2026). An optic-disc segmentation paper freezes the RETFound encoder, removes the class token and MLP classifier, and attaches a Segmenter-style mask-transformer decoder for dense prediction (Zhao et al., 15 Aug 2025). These studies establish RETFound not merely as a classifier, but as a reusable retinal representation backbone that can be paired with uncertainty heads, fusion modules, ordinal heads, and segmentation decoders.
4. Retinal imaging performance across task families
RETFound performs strongly on several retinal tasks, but its advantage is not uniform across all settings. In real-world OCT classification, RETFound is a competitive closed-set baseline. In the FMUE study on 11 retinal conditions from OCT, RETFound achieved an internal average F1 of 95.44% on a 19,046-image test set and an external overall accuracy of 88.08% across five public OCT datasets totaling 7,416 images (Peng et al., 2024). In OCT/OCTA retinal layer segmentation, the TAPE paper reports that RETFound-TAPE reaches 93.86 mDice and 89.22 mIoU on OCTA-500, outperforming MAE-TAPE and the trained-from-scratch baselines evaluated there (Su et al., 6 Apr 2026). For optic-disc segmentation, the first RETFound-based dense-prediction study reports about 96% Dice consistently across IDRID, Drishti-GS, RIM-ONE-r3, REFUGE, and GoDARTS, with strong domain-generalization and domain-adaptation results (Zhao et al., 15 Aug 2025).
In fundus-based disease classification, the picture is more mixed. On the large synthetic SynFundus-1M benchmark for multi-label classification of eleven retinal diseases, RETFound reaches macro-F1 0.9134 and macro-AUC , which is strong but below the best single architecture, ConvNeXtV2-Base, and below the final stacked XGBoost ensemble (Cao-Xue et al., 21 Aug 2025). An ablation in that study shows that retinal-domain pretraining still helps RETFound internally: the pretrained model reaches macro-AUC 0.9959 versus 0.9924 when trained from scratch (Cao-Xue et al., 21 Aug 2025).
Fine-grained retinal tasks expose clearer weaknesses. In fundus-based DME detection, RETFound does not consistently outperform lightweight CNNs. When trained on IDRiD, RETFound is best on external OEFI with AUC-PR 0.965 and AUC-ROC 0.939, but on IDRiD in-domain, on IDRiD-to-MESSIDOR-2 transfer, and in all experiments trained on MESSIDOR-2, it lags behind EfficientNet-B0 and often behind FLAIR; linear probing on frozen RETFound features is especially poor (Arellano et al., 8 Oct 2025). For early Alzheimer’s disease prediction from OCT B-scans, RETFound-C reaches mAUC and RETFound-S , both below ResNet-34 at , indicating that OCT-specific pretraining alone is insufficient for that small, weak-signal prediagnostic classification problem (Turkan et al., 7 Nov 2025).
5. Oculomics, multimodal prediction, and deployment uses
RETFound has been used not only for ocular disease detection but also for oculomics and multimodal clinical modeling. The review literature explicitly states that, after fine-tuning, RETFound can support detection and quantification of diabetic retinopathy and age-related macular degeneration and, with minimal fine-tuning, prediction of heart failure, stroke, and Parkinson’s disease; the same review also cites myocardial infarction in its discussion of downstream systemic prediction (Veldhuizen et al., 10 Jun 2025). This broader positioning has made RETFound central to the literature on retinal biomarkers of systemic disease.
HyMNet illustrates one concrete multimodal use. In that study, RETFound serves as the retinal backbone for hypertension prediction from fundus photographs, either as a unimodal classifier or as the FundusPath in a jointly trained multimodal network with age and gender. The unimodal RETFound model achieves F1 and AUC , while the multimodal HyMNet model improves to F1 and AUC (Baharoon et al., 2023). The same paper also notes that diabetes is a confounding variable in that cohort, which complicates straightforward interpretation of RETFound’s hypertension-related retinal signal (Baharoon et al., 2023).
Several comparative studies reinforce RETFound’s usefulness for systemic prediction, especially under limited labels. A head-to-head study versus DINOv2 reports that RETFound is superior to all tested DINOv2 variants in predicting heart failure, myocardial infarction, and ischaemic stroke from retinal images, with AUROCs spanning 0.732–0.796 for RETFound versus 0.663–0.771 for DINOv2, and that these trends persist even when only 10% of the fine-tuning data are used (Hou et al., 10 Feb 2025). Another study comparing RETFound with ResNet50, ViT-base, and SwinV2 concludes that traditional models are mostly comparable to RETFound for ocular disease detection with large datasets, but that RETFound is superior for systemic disease detection in smaller-data settings, including low-data diabetes, hypertension, and CKD experiments (Yew et al., 21 Jan 2025).
RETFound has also been used in deployment-oriented systems. In a two-tier edge–cloud cascade for diabetic retinopathy screening in rural settings, a MobileNetV3-small edge model performs referable/non-referable triage and forwards positive cases to a cloud RETFound-DINOv2 grader. The cloud-only RETFound baseline yields 80.76% accuracy and 0.8184 quadratic weighted kappa in the deployed 4-class space, while the full cascade preserves nearly the same grading performance, at 80.49% accuracy and 0.8167 kappa, while reducing cloud calls by 50.48% (Doshi et al., 13 May 2026). This shows RETFound functioning not as a universal first-pass model, but as a high-capacity specialist invoked selectively inside a resource-constrained clinical architecture.
6. Reliability, fairness, and the specialist-versus-generalist debate
A recurring criticism of RETFound concerns reliability under open-set clinical conditions. The FMUE paper makes this point explicitly: RETFound is a strong closed-set OCT classifier, but in that study it has no uncertainty estimation, no thresholded rejection mechanism, and no OOD detection capability. On ambiguous target-category images and non-target-category examples, RETFound is described as making incorrect predictions without warning of unreliability, whereas the uncertainty-aware extensions produce high uncertainty scores for referral (Peng et al., 2024). In that sense, the limitation emphasized is not raw classification accuracy but the absence of calibrated abstention behavior for real-world deployment.
Fairness findings are also nuanced. A study on BRSET reports that RETFound can reduce the gap between the maximum and minimum subgroup AUCs across gender and age relative to a ViT-L trained from scratch, and substantially improves worst-subgroup AUC at full data (Queiroz et al., 2024). However, the same paper shows that in data-efficient generalization, especially under reduced labeled data, age-related fairness gaps can widen again. The paper’s main warning is therefore that RETFound is not inherently bias-free: average label efficiency does not guarantee fair generalization under low-data adaptation (Queiroz et al., 2024).
The comparative literature does not deliver a single verdict on whether RETFound is universally preferable to large generalist or compact conventional models. One head-to-head study reports that DINOv2-Large outperforms RETFound on several ocular disease detection tasks, including diabetic retinopathy and multi-class CFP disease classification, while RETFound remains better for systemic disease prediction (Hou et al., 10 Feb 2025). A separate systematic study argues that large 303M-parameter RETFound models justify their cost only for challenging diabetic retinopathy grading, while compact 27–29M ImageNet-pretrained models are sufficient and often better on OCT, DME, and glaucoma classification (Isztl et al., 27 Nov 2025). By contrast, a later specialist-versus-generalist benchmark reports that RETFound-DINOv2 remains the best overall model across ocular disease detection and oculomics, with average fine-tuning AUROC 0.830 versus 0.816 for DINOv3-ViT-large and better external systemic AUROC, 0.599 versus 0.549–0.571 (Zhou et al., 3 Sep 2025). Another adaptation study shows that DINOv2-derived DINORET and block-expanded BE DINORET often surpass RETFound in frozen-backbone experiments and few-shot data efficiency, especially when embedding quality rather than end-to-end tuning is emphasized (Zoellin et al., 2024).
The most defensible synthesis is therefore task dependence. RETFound is consistently strong, sometimes state of the art, and often a difficult baseline to beat. But the literature does not support the stronger claim that retinal-domain pretraining always dominates either compact conventional models or very large generalist foundation models.
7. Derivatives, successors, and ecosystem influence
RETFound has become the reference point from which later ophthalmic foundation models differentiate themselves. The review literature identifies several direct successors or extensions. RETFound-Green is presented as a more efficient derivative that uses a ViT-Small backbone, 75K retinal images, a token-reconstruction objective, and 22.2M parameters, and is described as achieving comparable performance with half the data and 400 times less compute than earlier RETFound-family models (Engelmann et al., 2024, Veldhuizen et al., 10 Jun 2025). DERETFound is described as a ViT-Large MAE-based model using 150K real images plus 1M synthetic generated images and incorporating stable diffusion and image text-tagging to inject medical expertise (Veldhuizen et al., 10 Jun 2025). EyeFound initializes from RETFound weights and expands to 2.8M images and a broader ophthalmic modality set, while VisionFM broadens the modality scope further to CFP, FFA, OCTA, OCT, slit-lamp, B-scan ultrasound, and related tasks (Veldhuizen et al., 10 Jun 2025).
Other works define themselves partly by what RETFound omits. OCTCube argues that existing retinal foundation models, including RETFound, remain fundamentally 2D for OCT and therefore overlook the 3D structure of OCT volumes; OCTCube responds by directly modeling full 3D OCT volumes with 3D masked autoencoding (Liu et al., 2024). VOLMO, an ophthalmology-specific multimodal large model, uses RETFound only as a discriminative classification baseline and contrasts its task-specific fine-tuning requirement with VOLMO’s unified multimodal reasoning and generation workflow (Qin et al., 25 Mar 2026). In practical benchmarking, RETFound is thus no longer only a model but also an architectural baseline, a transfer-learning starting point, and a conceptual foil for newer designs.
Across the literature, RETFound’s enduring significance is twofold. First, it established the retinal foundation-model paradigm in ophthalmology: large-scale self-supervised retinal pretraining, ViT-based representations, and label-efficient adaptation across ocular and systemic tasks (Veldhuizen et al., 10 Jun 2025). Second, it created a common benchmark against which later work measures efficiency, generalization, multimodality, uncertainty handling, fairness, and task breadth. RETFound therefore remains central not because every later study finds it optimal, but because subsequent ophthalmic foundation-model research is still organized around the question of when RETFound-style retinal specialization is sufficient, necessary, or worth extending.