DeepSeeNet: Automated AMD Severity Classification

• DeepSeeNet is a deep learning system that automates patient-level AMD severity grading by reconstructing the AREDS Simplified Severity Scale using bilateral fundus photographs.
• It employs three specialized sub-networks (D-Net, P-Net, LA-Net) based on Inception-v3 to detect drusen, pigmentary abnormalities, and late AMD.
• Fine-tuning pretrained on ImageNet improved its accuracy and interpretability, surpassing retinal specialist performance in several severity classes.

Searching arXiv for DeepSeeNet and closely related papers to ground the article in the literature. DeepSeeNet is a deep learning system for automated, patient-level classification of age-related macular degeneration (AMD) severity from bilateral color fundus photographs (CFP). Its defining feature is that it does not simply classify each eye independently; instead, it reproduces the clinical logic of the AREDS Simplified Severity Scale by identifying key lesion types in each eye and then combining bilateral findings into a single patient score from 0 to 5 (Peng et al., 2018). In the literature around the model, DeepSeeNet also functions as a methodological lineage: later work reused its transfer-learning and fundus-photo-based framework for geographic atrophy (GA) and central geographic atrophy (CGA) detection (Keenan et al., 2019), while other work positioned itself as a multi-task refinement of the same clinically structured grading philosophy for the AREDS 9-step non-advanced AMD scale (Chen et al., 2018). More recent application papers have incorporated DeepSeeNet-derived AMD outputs into broader monitoring systems, but such papers do not constitute new DeepSeeNet methods papers (Reid et al., 20 Sep 2025).

1. Clinical task and AREDS-based formulation

DeepSeeNet addresses a specifically patient-based problem: estimation of AMD severity and progression risk using images from both eyes rather than isolated eye-level prediction (Peng et al., 2018). This distinction is clinically consequential because the fellow eye contributes to the AREDS Simplified Severity Scale and therefore to the patient’s estimated risk of progression to late AMD.

The model follows a modified version of the AREDS Simplified Severity Scale with scores from 0 to 5. Scores 0 to 4 are assigned from the presence of two risk factors in each eye: large drusen with diameter $>125\,\mu m$, and AMD pigmentary abnormalities at the macula. Score 5 is assigned if late AMD is present in either eye, where late AMD is defined as neovascular AMD or central geographic atrophy (Peng et al., 2018). The paper gives the patient-level rule as

$$S = \begin{cases} 5, & \text{if } L_L = 1 \text{ or } L_R = 1 \ D_L + P_L + D_R + P_R, & \text{otherwise} \end{cases}$$

with $D_e$ indicating large drusen, $P_e$ pigmentary abnormalities, and $L_e$ late AMD in eye $e \in \{L,R\}$ (Peng et al., 2018).

The clinical meaning of scores 0 to 4 is tied to 5-year risk of developing late AMD in at least one eye. The cited risks are 0.4% for score 0, 3.1% for score 1, 11.8% for score 2, 25.9% for score 3, and 47.3% for score 4; score 5 means late AMD is already present in at least one eye (Peng et al., 2018). This framing explains why DeepSeeNet was designed around bilateral, patient-level severity rather than direct binary referral classification.

A central methodological point is that DeepSeeNet simulates the human grading process. It first detects drusen size, pigmentary abnormalities, and late AMD for each eye, then computes the patient score algorithmically from those outputs (Peng et al., 2018). This structured decomposition is the source of its relative transparency: the final score is not a monolithic end-to-end label, but the consequence of identifiable intermediate lesion predictions.

2. Dataset, labels, and preprocessing

The original study used the AREDS dataset, comprising stereoscopic color fundus photographs with a 30° imaging field centered at the fovea (Peng et al., 2018). Because a stereoscopic pair contains redundant information, the study used one image per eye, generally the left image of the pair and the right image only if the left was missing, approximately $0.5\%$ of the time (Peng et al., 2018).

The total dataset size was 59,302 fundus images from 4,549 participants. The participant-disjoint split consisted of a training set of 58,402 images from 4,099 participants and a test set of 900 images from 450 participants, where the test set used bilateral baseline images only because those baseline images also had retinal specialist assessments available (Peng et al., 2018). Images from the 450 test participants at visits other than baseline were excluded from both train and test (Peng et al., 2018).

Ground-truth labels were Reading Center grades from the University of Wisconsin Reading Center. The grading workflow was structured: a senior grader performed preliminary AMD grading, a junior grader performed detailed lesion grading, a computerized algorithm extracted AMD severity from the detailed grades, and disagreements were adjudicated by a senior investigator (Peng et al., 2018). The paper emphasizes independent masked grading, experienced graders, inter-grader and intra-grader agreement monitoring, and annual masked re-grading to detect temporal drift (Peng et al., 2018). This means the gold standard was a reading-center protocol rather than ad hoc individual clinician annotation.

Preprocessing was intentionally limited. The reported steps were to crop the fundus image to a square macula-centered field and resize it to $224 \times 224$ pixels (Peng et al., 2018). The authors explicitly note that they did not perform extensive preprocessing such as retinal boundary detection, color normalization, or local illumination normalization (Peng et al., 2018). This design choice was presented as a way to improve generalizability, although it also created vulnerability to low-quality images.

The baseline participant distribution by patient-level AREDS simplified score was imbalanced. In the training participants, score 0 comprised 1,258 cases, score 1 comprised 653, score 2 comprised 461, score 3 comprised 303, score 4 comprised 279, and score 5 comprised 537. In the testing participants, score 0 comprised 185, score 1 79, score 2 56, score 3 46, score 4 33, and score 5 51 (Peng et al., 2018). The paper also reports eye-level lesion prevalence distributions for drusen, pigmentary abnormalities, and late AMD in both train and test cohorts, which are relevant to interpreting class imbalance and the comparative weakness of the late-AMD detector (Peng et al., 2018).

3. Architecture and training strategy

DeepSeeNet is a two-stage, clinically structured system composed of three sub-networks: D-Net for drusen classification, P-Net for pigmentary abnormality detection, and LA-Net for late AMD detection (Peng et al., 2018). D-Net classifies drusen into small/none, medium, and large; P-Net is a binary classifier for pigmentary abnormalities; LA-Net is a binary classifier for late AMD, defined as neovascular AMD or central geographic atrophy (Peng et al., 2018).

All three subnetworks use Inception-v3. The paper describes Inception-v3 as a state-of-the-art CNN for image classification with total depth of 317 layers and more than 21 million learnable weights (Peng et al., 2018). Each eye image is processed by the relevant subnetworks to estimate drusen category, pigmentary abnormality, and late AMD status. These per-eye outputs are then combined across both eyes into a patient-level severity estimate. In the paper’s notation,

$$\hat S = \begin{cases} 5, & \hat L_L = 1 \text{ or } \hat L_R = 1 \ \hat D_L + \hat P_L + \hat D_R + \hat P_R, & \text{otherwise} \end{cases}$$

where $\hat D_e = \mathbf{1}\{\hat d_e = \text{large}\}$, $$S = \begin{cases} 5, & \text{if } L_L = 1 \text{ or } L_R = 1 \ D_L + P_L + D_R + P_R, & \text{otherwise} \end{cases}$$0, and $$S = \begin{cases} 5, & \text{if } L_L = 1 \text{ or } L_R = 1 \ D_L + P_L + D_R + P_R, & \text{otherwise} \end{cases}$$1 (Peng et al., 2018).

The study compared three training strategies. The best-performing method was Fine-tuned DeepSeeNet, which started with ImageNet-pretrained Inception-v3 and fine-tuned all layers on AREDS images. A second approach, described as an MLP strategy or fixed feature extractor, froze most layers of ImageNet-pretrained Inception-v3 and trained only the last two dense layers of 256 and 128 units. A third approach, Fully-trained DeepSeeNet, started from randomly initialized Inception-v3 and trained all layers on AREDS only (Peng et al., 2018). The reported interpretation was that ImageNet pretraining provided useful low-level initialization, but full fine-tuning was preferable to using Inception merely as a frozen feature extractor because retinal images differ substantially from natural images (Peng et al., 2018).

Training details were explicit on several points: Keras with TensorFlow backend, Adam optimizer, learning rate $$S = \begin{cases} 5, & \text{if } L_L = 1 \text{ or } L_R = 1 \ D_L + P_L + D_R + P_R, & \text{otherwise} \end{cases}$$2, mini-batch size 32, input resolution $$S = \begin{cases} 5, & \text{if } L_L = 1 \text{ or } L_R = 1 \ D_L + P_L + D_R + P_R, & \text{otherwise} \end{cases}$$3, and training stopped after 5 epochs when accuracy no longer increased or began to decrease (Peng et al., 2018). Hardware consisted of 32 Intel Xeon CPUs, an NVIDIA GeForce GTX 1080 Ti with 11 GB, and 512 GB RAM (Peng et al., 2018). The paper does not explicitly name the loss function; standard cross-entropy objectives are plausible, but that is not printed as a formal methodological statement in the source text (Peng et al., 2018).

4. Evaluation results and comparative performance

Evaluation used overall accuracy, sensitivity, specificity, Cohen’s kappa, and ROC curves/AUC (Peng et al., 2018). Confidence intervals were obtained by sampling 450 patients with replacement from the test set, evaluating performance, repeating 2,000 times, and reporting 95% confidence intervals (Peng et al., 2018). The comparison benchmark was unusually strong: 88 retinal specialists who had served as AREDS investigators independently assessed the 450 baseline participants as part of a qualification survey, and those clinical assessments were converted into the same patient-level AREDS Simplified Severity Scale for comparison against Reading Center grades (Peng et al., 2018).

The best-performing strategy was Fine-tuned DeepSeeNet. Its patient-based AMD severity classification performance on the 450-patient test set was accuracy 0.671, sensitivity 0.590, specificity 0.930, and Cohen’s kappa 0.558, compared with retinal specialists at accuracy 0.599, sensitivity 0.512, specificity 0.916, and Cohen’s kappa 0.467 (Peng et al., 2018). The alternative strategies performed worse: the MLP strategy achieved accuracy 0.436 and kappa 0.163, while Fully-trained DeepSeeNet achieved accuracy 0.624 and kappa 0.487 (Peng et al., 2018).

The lesion-specific subnetworks showed strong discriminative performance. D-Net for large drusen detection achieved accuracy 0.742, sensitivity 0.718, specificity 0.871, kappa 0.601, and AUC 0.94; P-Net for pigmentary abnormalities achieved accuracy 0.890, sensitivity 0.732, specificity 0.957, kappa 0.723, and AUC 0.93; LA-Net for late AMD achieved accuracy 0.967, sensitivity 0.627, specificity 0.987, kappa 0.663, and AUC 0.97 (Peng et al., 2018). The paper’s interpretation is asymmetrical: DeepSeeNet was better than specialists for large drusen and pigmentary abnormalities, but for late AMD it had very high specificity and high AUC while still showing lower sensitivity and lower kappa than specialists (Peng et al., 2018).

Class-wise patient-level performance clarifies the same pattern. DeepSeeNet exceeded specialists for true scores 0 through 4, with class accuracies of 90.8%, 43.0%, 48.2%, 34.8%, and 72.7%, compared with specialists at 83.8%, 29.1%, 39.3%, 26.1%, and 45.5%, respectively. For score 5, however, specialists were better at 82.4% versus DeepSeeNet’s 64.7% (Peng et al., 2018). The paper notes from confusion matrices that, for true scores 0 to 4, DeepSeeNet’s errors were usually off by only one scale step (Peng et al., 2018).

A later paper built using the DeepSeeNet methods extended this framework to binary lesion detection for geographic atrophy and central geographic atrophy from AREDS color fundus photographs (Keenan et al., 2019). That study used 59,812 color fundus photographs from 4,582 participants, trained three binary classifiers with Inception-v3 and ImageNet pretraining, and reported AUCs of 0.933–0.976 for GA detection, 0.939–0.976 for CGA detection from all eyes, and 0.827–0.888 for centrality detection from GA eyes (Keenan et al., 2019). The paper states that these models were built using the methods described in DeepSeeNet and that the code and pretrained models were publicly available in the DeepSeeNet repository (Keenan et al., 2019). This places the GA/CGA work within a DeepSeeNet-family framework rather than as an independent unrelated system.

5. Interpretability, error modes, and limitations

Interpretability in DeepSeeNet is primarily structural. Because the model first predicts drusen category, pigmentary abnormality, and late AMD for each eye before computing the patient score, a clinician can inspect which lesions in which eye drove the final severity output (Peng et al., 2018). This is a stronger form of semantic decomposition than direct image-to-score classification.

The original paper also used t-distributed stochastic neighbor embedding on the 128-dimensional vector from the last dense layer. It reported that, for drusen, small/none and large drusen classes were separated while medium drusen lay between them and showed more overlap; for pigmentary abnormalities and late AMD, positive and negative classes were clearly separated (Peng et al., 2018). This supports the view that the learned representation tracks clinically meaningful lesion structure, and it also helps explain why intermediate drusen categories are difficult.

Image-specific class saliency maps were generated to highlight pixels contributing most to the predictions. The reported finding was that, for drusen, highlighted regions corresponded to visible drusen; for pigmentary abnormality, they aligned with relevant pigment changes; and for late AMD, highlighted areas corresponded to visible advanced lesions (Peng et al., 2018). The paper treats this as evidence that the model was largely attending to clinically meaningful image regions, while also noting that saliency maps do not fully explain internal decision logic.

The qualitative error analysis identifies several recurrent failure modes. Among participants incorrectly classified as having late AMD, 50% had noncentral GA in at least one eye (Peng et al., 2018). This matters because, in the study’s labeling framework, noncentral GA was not counted as late AMD even though it is visually and biologically close to that category. Of the participants incorrectly classified as late AMD, 25% had digital artifacts obscuring the fovea; pale retinas, brightness issues, and digital artifacts could also cause images to be misclassified as geographic atrophy (Peng et al., 2018). The paper therefore explicitly ties some errors to image quality and suggests future quality filtering or corrective preprocessing.

The limitations emphasized in the original study are class imbalance, especially for late AMD; reliance on a single modality, color fundus photographs, without OCT or fundus autofluorescence; sensitivity to image quality; intentionally limited preprocessing; and the need for external validation beyond AREDS (Peng et al., 2018). The study also notes that AREDS began in 1992, so the image acquisition era differs from modern digital retinal imaging (Peng et al., 2018). A plausible implication is that DeepSeeNet combines strong internal validity within a carefully graded clinical-trial image archive with uncertain transfer properties across newer devices, broader pathology mixes, and screening-style acquisition settings.

6. Successors, methodological relatives, and application-level reuse

DeepSeeNet generated a distinct methodological lineage. One direct successor was a multi-task deep learning model for automated classification of AMD on the AREDS 9-step non-advanced severity scale (Chen et al., 2018). That work was explicitly built on previous DeepSeeNet and replaced separate single-task submodels with a shared multi-task architecture predicting four AMD characteristics in parallel: drusen area, geographic atrophy, increased pigment, and depigmentation (Chen et al., 2018). The model used 10 Inception-V3 blocks as a shared feature extractor, followed by Global Average Pooling, a dense layer with 1024 units, dropout, and task-specific heads composed of dense layers with 256 and 128 units plus softmax output (Chen et al., 2018). Its principal claim was that joint learning of related AMD characteristics improved performance and robustness, especially on AREDS2 as an independent test set (Chen et al., 2018).

The relationship among the main DeepSeeNet-related papers can be summarized concisely.

Paper	Relation to DeepSeeNet	Primary task
"DeepSeeNet: A deep learning model for automated classification of patient-based age-related macular degeneration severity from color fundus photographs" (Peng et al., 2018)	Canonical DeepSeeNet paper	Patient-based AMD severity classification on the AREDS Simplified Severity Scale
"A multi-task deep learning model for the classification of Age-related Macular Degeneration" (Chen et al., 2018)	Built on previous DeepSeeNet	AREDS 9-step non-advanced AMD grading via multi-task learning
"A deep learning approach for automated detection of geographic atrophy from color fundus photographs" (Keenan et al., 2019)	Built using DeepSeeNet methods; code in DeepSeeNet repository	GA and CGA detection from color fundus photographs
"Development of a Mobile Application for at-Home Analysis of Retinal Fundus Images" (Reid et al., 20 Sep 2025)	Application-level integration of adopted DeepSeeNet outputs	Longitudinal monitoring of retinal metrics in a mobile app

Application-level reuse requires caution. In the 2025 mobile monitoring paper, DeepSeeNet appears as an adopted AMD-analysis component inside a broader retinal fundus image monitoring platform rather than as the object of methodological development (Reid et al., 20 Sep 2025). The paper states that “The DeepSeeNet model was used to output information related to AMD” and attributes to it an accuracy of 0.671, sensitivity of 0.590, and specificity of 0.930, with retinal specialists cited as having accuracy 0.599, sensitivity 0.512, and specificity 0.916; those are prior-model performance characteristics, not outcomes of a new DeepSeeNet experiment in that study (Reid et al., 20 Sep 2025). The meaningful DeepSeeNet-derived outputs presented in the app are Age-Related Macular Degeneration Grade, Drusen Score, Pigmentary abnormalities, Geographic atrophy, and Central Geographic Atrophy (Reid et al., 20 Sep 2025).

That same mobile paper contains an internal inconsistency: in the literature review, DeepSeeNet is correctly described as an AMD model that classifies fundus images with the AREDS Simplified Severity Scale, but later one sentence refers to “a model to detect glaucoma” from DeepSeeNet (Reid et al., 20 Sep 2025). The surrounding output descriptions remain AMD-related rather than glaucoma-related, and the paper does not define a coherent DeepSeeNet glaucoma score or provide glaucoma-specific validation (Reid et al., 20 Sep 2025). A careful technical reading therefore indicates that DeepSeeNet’s role in that system is AMD monitoring, not glaucoma detection.

Taken together, these later papers show that DeepSeeNet is both a specific model and an architectural template. In its canonical form, it is an Inception-v3-based, transfer-learning system for bilateral, patient-level AMD severity grading under the AREDS Simplified Severity Scale (Peng et al., 2018). In the surrounding literature, it also functions as a reusable framework for more specialized AMD-related tasks such as GA/CGA detection (Keenan et al., 2019), as a methodological reference point for multi-task AMD grading systems (Chen et al., 2018), and as an adopted component in longitudinal retinal monitoring applications that present AMD-related metrics over time without claiming standalone diagnosis from a single at-home image (Reid et al., 20 Sep 2025).