RETFound-DINOv2: Specialist Retinal Model

• The paper introduces RETFound-DINOv2, which integrates DINOv2 self-distillation with retinal-specific pre-training to achieve higher AUROC across ten ocular tasks.
• RETFound-DINOv2 is built on a ViT-Large backbone, pre-trained on 904K fundus images using dedicated fundus preprocessing techniques to enhance feature granularity.
• Fine-tuning yields a 3–4 point AUROC improvement over linear probing, underscoring a trade-off between performance gains and increased computational cost.

RETFound-DINOv2 is a specialist retinal vision foundation model developed to test whether domain-specific pre-training remains necessary in the presence of increasingly large generalist vision foundation models such as DINOv2 and DINOv3. In the comparative study "Generalist versus Specialist Vision Foundation Models for Ocular Disease and Oculomics," RETFound-DINOv2 is evaluated against DINOv2-ViT-giant, DINOv3-ViT-large, and RETFound-MAE on ocular disease detection and oculomics tasks, using both fine-tuning and linear probing. The reported results show that RETFound-DINOv2 consistently outperforms the generalist baselines and RETFound-MAE on the ten-task aggregate, while also exhibiting stronger generalisability and data efficiency (Zhou et al., 3 Sep 2025).

1. Position within retinal foundation modeling

RETFound-DINOv2 occupies a specific position in the evolution of retinal foundation models: it preserves a retinal-specialist training regime while adopting the self-distillation framework associated with DINOv2. The central research question is whether the scale of generalist pre-training can substitute for domain-specific retinal pre-training, or whether a measurable specialist advantage persists in ophthalmic applications (Zhou et al., 3 Sep 2025).

This question arose against an empirical background in which generalist and specialist models had shown complementary strengths. In the earlier head-to-head study "Is an Ultra Large Natural Image-Based Foundation Model Superior to a Retina-Specific Model for Detecting Ocular and Systemic Diseases?," DINOv2-large outperformed RETFound in detecting diabetic retinopathy and multi-class eye diseases, and DINOv2-base outperformed RETFound in glaucoma, whereas RETFound remained superior for heart failure, myocardial infarction, and ischaemic stroke prediction (Hou et al., 10 Feb 2025). RETFound-DINOv2 can therefore be understood as an attempt to combine DINO-style representation learning with retinal specialization rather than treating generalist and specialist paradigms as mutually exclusive.

A plausible implication is that RETFound-DINOv2 is best interpreted not simply as another RETFound variant, but as a specialist re-instantiation of a generalist self-supervised recipe under retinal data constraints. The reported findings support that interpretation: the gap between specialist and generalist models narrows with scale, but is not eliminated in the retinal setting (Zhou et al., 3 Sep 2025).

2. Pre-training data, preprocessing, and self-supervision

RETFound-DINOv2 was pre-trained on 904 K de-identified colour fundus images from the AlzEye cohort. Each image is a 256 × 256 crop that is subsequently resized and normalised, and no additional modalities (e.g. OCT) were used; all inputs are 2D fundus photographs (Zhou et al., 3 Sep 2025).

The preprocessing pipeline is explicitly fundus-specific. AutoMorph was applied for vessel segmentation and standard colour normalisation. Pre-training augmentations comprised two-view random cropping with scale 0.2–1.0 of original, random horizontal flipping, Gaussian blur, colour jitter, and Normalise(mean, std) in RGB space (Zhou et al., 3 Sep 2025). During DINO-style multi-crop training, the typical setting was 2 global crops + 8 local crops per image.

The learning objective follows DINOv2’s self-distillation framework. A student network $f_\theta$ and a momentum-updated teacher $f_{\theta'}$ are fed different random views of the same image, and the student output $p_{\mathrm{stu}} \in \mathbb{R}^K$ is trained to match the teacher output $p_{\mathrm{tea}} \in \mathbb{R}^K$ through the cross-entropy loss

$$L_{\mathrm{DINO}} = - \sum_{k=1}^K p_{\mathrm{tea},k} \cdot \log p_{\mathrm{stu},k}.$$

Here, $p_{\mathrm{stu}}$ and $p_{\mathrm{tea}}$ are the softmaxed patch-level embeddings with temperatures $T_{\mathrm{stu}}$ and $T_{\mathrm{tea}}$. The teacher parameters are updated at each step by momentum:

$\theta'_t = m\,\theta'_{t-1} + (1-m)\,\theta_t.$

The momentum coefficient $f_{\theta'}$0 typically ramps from 0.996 to 1.0 (Zhou et al., 3 Sep 2025).

This setup matters because the reported interpretation of RETFound-DINOv2’s advantage is representational rather than architectural: the model is specialized by retinal data and retinal preprocessing, not by introducing a new transformer mechanism.

3. Backbone architecture and specialist modifications

The backbone is a ViT-Large transformer with 24 Transformer encoder blocks, embedding dimension 1 024, 16 heads, and patch size 16 × 16, yielding sequence length 256. An MLP head is retained only for projection during pre-training and is discarded during downstream adaptation (Zhou et al., 3 Sep 2025).

The paper emphasizes that the specialist modifications are minimal at the architectural level. There are no changes to the core ViT architecture beyond initialising from DINOv2-ViT-Large weights. The specialization is instead imposed through the fundus-specific input pipeline; no new attention masks or positional embeddings were introduced (Zhou et al., 3 Sep 2025).

In the comparative tables, RETFound-DINOv2 is reported as a 300 M model, placing it directly against DINOv3-ViT-large (300 M) and against the substantially larger DINOv2-ViT-giant (1.1 B) in average ten-task evaluation (Zhou et al., 3 Sep 2025). This makes the comparison relevant to a common misconception that specialist superiority, if present, necessarily reflects larger capacity. In the reported experiments, the best-performing model is not the largest model, but the retinal-specialist ViT-Large.

4. Downstream adaptation protocols

Two downstream adaptation protocols were applied uniformly across ocular disease detection and oculomics tasks. In fine-tuning (FT), all ViT parameters and a new single-layer classifier head are optimized. In linear probing (LP), the entire ViT encoder is frozen and only the final classification head is trained, under the same augmentation and learning-rate schedule (Zhou et al., 3 Sep 2025).

For fine-tuning, the reported configuration is: batch size 24, 50 epochs, linear warm-up from 0 to $f_{\theta'}$1 over the first 10 epochs, followed by cosine decay to $f_{\theta'}$2 over the next 40 epochs (Zhou et al., 3 Sep 2025). Data were split at the patient level into 55%/15%/30% for internal train/validation/test to avoid visit bias, with the external evaluation performed on an entire held-out cohort. Adaptation augmentations reused the pre-training style: random crop 0.2–1.0, resize to 224 × 224, flip, and normalise (Zhou et al., 3 Sep 2025).

The earlier RETFound-versus-DINOv2 study used a different fine-tuning pipeline: a frozen ViT backbone with a task-specific MLP head, AdamW with weight decay 0.05, 100 epochs total, 10 epoch warm-up from 0 to $f_{\theta'}$3, 90 epochs cosine-anneal to $f_{\theta'}$4, and batch size 32 (Hou et al., 10 Feb 2025). That earlier protocol is historically relevant because it established the first generalist-versus-specialist retinal benchmark, whereas the later RETFound-DINOv2 study shifted the emphasis from fixed-backbone adaptation to a direct comparison of FT and LP across specialist and generalist models.

The reported conclusion on adaptation is operationally important: fine-tuning consistently edges out linear probing by $f_{\theta'}$5AUROC $f_{\theta'}$6–$f_{\theta'}$7 points, but at 2–3× higher computational cost, while linear probing remains attractive in resource-limited settings (Zhou et al., 3 Sep 2025).

5. Quantitative performance across ocular disease detection and oculomics

The principal reported metric is mean AUROC ± 95% CI over $f_{\theta'}$8 bootstrap samples. On the average across ten downstream tasks—5 ocular and 5 oculomics—RETFound-DINOv2 achieves the best result among the compared models (Zhou et al., 3 Sep 2025).

Model	Avg AUROC	Wilcoxon p-value vs RETFound-DINOv2
DINOv2-ViT-giant (1.1 B)	0.800 ± 0.004	< 0.01
DINOv3-ViT-large (300 M)	0.816 ± 0.003	< 0.05
RETFound-DINOv2 (300 M)	0.830 ± 0.003	—
RETFound-MAE (300 M)	0.809 ± 0.004	< 0.05

The paper further states that RETFound-DINOv2 outperforms the best generalist baseline, DINOv3-ViT-large, by approximately 1.4% in Avg AUROC with $f_{\theta'}$9, and that Wilcoxon signed-rank tests over per-task AUROC confirm the robustness of that edge (Zhou et al., 3 Sep 2025).

Per-task highlights for RETFound-DINOv2 (FT) include diabetic retinopathy on MESSIDOR2: 0.892 ± 0.005, glaucoma on Papila: 0.845 ± 0.006, multi-disease on the Retina dataset: 0.812 ± 0.007, myocardial infarction prediction: 0.619 ± 0.010, and stroke prediction: 0.573 ± 0.012 (Zhou et al., 3 Sep 2025). On external evaluation on UK Biobank, after fine-tuning on AlzEye, RETFound-DINOv2 achieves Avg AUROC = 0.599 ± 0.009, compared with RETFound-MAE 0.571, DINOv2-G 0.560, and DINOv3-L 0.549 (Zhou et al., 3 Sep 2025).

The classification operating point was selected at max $p_{\mathrm{stu}} \in \mathbb{R}^K$0. Averaged across ten tasks in fine-tuning mode on the test set, the reported values are Sensitivity = 0.79 ± 0.02 and Specificity = 0.75 ± 0.02 (Zhou et al., 3 Sep 2025). These numbers indicate that the model’s advantage is not restricted to rank-based discrimination alone; it also translates into a stated sensitivity/specificity trade-off under a fixed selection rule.

The earlier comparative benchmark provides useful contrast. In that study, DINOv2-Large exceeded RETFound in several ocular tasks, including APTOS-2019: 0.952 (0.950–0.954) vs 0.944 (0.941–0.946), MESSIDOR-2: 0.906 (0.902–0.910) vs 0.883 (0.878–0.889), and Retina CFP multi-class disease: 0.892 (0.883–0.902) vs 0.846 (0.836–0.856), while RETFound led in systemic prediction tasks such as heart failure: 0.796 (0.767–0.827) and ischaemic stroke: 0.754 (0.684–0.838) (Hou et al., 10 Feb 2025). This contrast clarifies the significance of RETFound-DINOv2: it is not merely another retinal specialist, but a model that closes the earlier ocular-performance deficit while retaining the specialist orientation associated with oculomics.

6. Data efficiency, computational trade-offs, and representational interpretation

RETFound-DINOv2’s reported advantage extends beyond full-data performance. In the data-efficiency analysis, with only 10% of training data, RETFound-DINOv2 (FT) attains AUROC = 0.720, corresponding to 92.6% of full-data performance. Among the compared models, DINOv3-L and RETFound-DINOv2 show the best retention of performance in low-data regimes (Zhou et al., 3 Sep 2025). The paper therefore links specialization not only to peak AUROC but also to label efficiency, a practically important property in medical imaging where annotated datasets are often limited.

Adaptation efficiency was quantified on APTOS-2019 (3 662 images) for fine-tuning:

Model	GPU memory / runtime
DINOv2-G	67 600 MiB / 106 min
DINOv3-L	34 800 MiB / 59.8 min
RETFound-DINOv2	24 400 MiB / 34.8 min
RETFound-MAE	21 900 MiB / 20.0 min

In the same analysis, linear probing reduces memory by ~10–20% and runtime by ~50% (Zhou et al., 3 Sep 2025). A plausible implication is that RETFound-DINOv2 occupies an intermediate computational position: it is more expensive than RETFound-MAE, but materially less demanding than the largest generalist baselines while delivering the best average predictive performance.

The paper also provides a representational interpretation through feature self-similarity. RETFound-DINOv2 has feature self-similarity average 0.798 versus 0.915 for DINOv3-L, and the authors explicitly interpret lower self-similarity as higher feature granularity (Zhou et al., 3 Sep 2025). This suggests that the model’s specialist advantage is associated with finer-grained retinal feature discrimination rather than with architectural novelty.

The broader conclusion is deliberately non-absolutist. The study states that specialist retinal foundation models remain the most effective choice for clinical applications, yet also notes that the narrowing gap with generalist foundation models indicates that continued scaling of data and model size can produce domain-relevant gains (Zhou et al., 3 Sep 2025). The stated future direction is a hybrid approach: leveraging large-scale generalist pre-training followed by domain-specific retinal tuning, with the expectation that such a strategy may yield favorable performance-efficiency trade-offs in clinical ophthalmology.