---
title: 'DeepSeeNet: Automated AMD Severity Classification'
url: https://www.emergentmind.com/topics/deepseenet
type: topic
---

# DeepSeeNet: Automated AMD Severity Classification

Searching arXiv for DeepSeeNet and closely related papers to ground the article in the literature.
DeepSeeNet is a deep learning system for automated, patient-level classification of age-related macular degeneration (AMD) severity from bilateral color fundus photographs (CFP). Its defining feature is that it does not simply classify each eye independently; instead, it reproduces the clinical logic of the AREDS Simplified Severity Scale by identifying key lesion types in each eye and then combining bilateral findings into a single patient score from 0 to 5 [1811.07492]. In the literature around the model, DeepSeeNet also functions as a methodological lineage: later work reused its transfer-learning and fundus-photo-based framework for geographic atrophy (GA) and central geographic atrophy (CGA) detection [1906.03153], while other work positioned itself as a multi-task refinement of the same clinically structured grading philosophy for the AREDS 9-step non-advanced AMD scale [1812.00422]. More recent application papers have incorporated DeepSeeNet-derived AMD outputs into broader monitoring systems, but such papers do not constitute new DeepSeeNet methods papers [2509.16814].

## 1. Clinical task and AREDS-based formulation

DeepSeeNet addresses a specifically patient-based problem: estimation of AMD severity and progression risk using images from both eyes rather than isolated eye-level prediction [1811.07492]. This distinction is clinically consequential because the fellow eye contributes to the AREDS Simplified Severity Scale and therefore to the patient’s estimated risk of progression to late AMD.

The model follows a modified version of the AREDS Simplified Severity Scale with scores from 0 to 5. Scores 0 to 4 are assigned from the presence of two risk factors in each eye: large drusen with diameter \(>125\,\mu m\), and AMD pigmentary abnormalities at the macula. Score 5 is assigned if late AMD is present in either eye, where late AMD is defined as neovascular AMD or central geographic atrophy [1811.07492]. The paper gives the patient-level rule as
\[
S =
\begin{cases}
5, & \text{if } L_L = 1 \text{ or } L_R = 1 \\
D_L + P_L + D_R + P_R, & \text{otherwise}
\end{cases}
\]
with \(D_e\) indicating large drusen, \(P_e\) pigmentary abnormalities, and \(L_e\) late AMD in eye \(e \in \{L,R\}\) [1811.07492].

The clinical meaning of scores 0 to 4 is tied to 5-year risk of developing late AMD in at least one eye. The cited risks are 0.4% for score 0, 3.1% for score 1, 11.8% for score 2, 25.9% for score 3, and 47.3% for score 4; score 5 means late AMD is already present in at least one eye [1811.07492]. This framing explains why DeepSeeNet was designed around bilateral, patient-level severity rather than direct binary referral classification.

A central methodological point is that DeepSeeNet simulates the human grading process. It first detects drusen size, pigmentary abnormalities, and late AMD for each eye, then computes the patient score algorithmically from those outputs [1811.07492]. This structured decomposition is the source of its relative transparency: the final score is not a monolithic end-to-end label, but the consequence of identifiable intermediate lesion predictions.

## 2. Dataset, labels, and preprocessing

The original study used the AREDS dataset, comprising stereoscopic color fundus photographs with a 30° imaging field centered at the fovea [1811.07492]. Because a stereoscopic pair contains redundant information, the study used one image per eye, generally the left image of the pair and the right image only if the left was missing, approximately \(0.5\%\) of the time [1811.07492].

The total dataset size was 59,302 fundus images from 4,549 participants. The participant-disjoint split consisted of a training set of 58,402 images from 4,099 participants and a test set of 900 images from 450 participants, where the test set used bilateral baseline images only because those baseline images also had retinal specialist assessments available [1811.07492]. Images from the 450 test participants at visits other than baseline were excluded from both train and test [1811.07492].

Ground-truth labels were Reading Center grades from the University of Wisconsin Reading Center. The grading workflow was structured: a senior grader performed preliminary AMD grading, a junior grader performed detailed lesion grading, a computerized algorithm extracted AMD severity from the detailed grades, and disagreements were adjudicated by a senior investigator [1811.07492]. The paper emphasizes independent masked grading, experienced graders, inter-grader and intra-grader agreement monitoring, and annual masked re-grading to detect temporal drift [1811.07492]. This means the gold standard was a reading-center protocol rather than ad hoc individual clinician annotation.

Preprocessing was intentionally limited. The reported steps were to crop the fundus image to a square macula-centered field and resize it to \(224 \times 224\) pixels [1811.07492]. The authors explicitly note that they did not perform extensive preprocessing such as retinal boundary detection, color normalization, or local illumination normalization [1811.07492]. This design choice was presented as a way to improve generalizability, although it also created vulnerability to low-quality images.

The baseline participant distribution by patient-level AREDS simplified score was imbalanced. In the training participants, score 0 comprised 1,258 cases, score 1 comprised 653, score 2 comprised 461, score 3 comprised 303, score 4 comprised 279, and score 5 comprised 537. In the testing participants, score 0 comprised 185, score 1 79, score 2 56, score 3 46, score 4 33, and score 5 51 [1811.07492]. The paper also reports eye-level lesion prevalence distributions for drusen, pigmentary abnormalities, and late AMD in both train and test cohorts, which are relevant to interpreting class imbalance and the comparative weakness of the late-AMD detector [1811.07492].

## 3. Architecture and training strategy

DeepSeeNet is a two-stage, clinically structured system composed of three sub-networks: D-Net for drusen classification, P-Net for pigmentary abnormality detection, and LA-Net for late AMD detection [1811.07492]. D-Net classifies drusen into small/none, medium, and large; P-Net is a binary classifier for pigmentary abnormalities; LA-Net is a binary classifier for late AMD, defined as neovascular AMD or central geographic atrophy [1811.07492].

All three subnetworks use Inception-v3. The paper describes Inception-v3 as a state-of-the-art CNN for image classification with total depth of 317 layers and more than 21 million learnable weights [1811.07492]. Each eye image is processed by the relevant subnetworks to estimate drusen category, pigmentary abnormality, and late AMD status. These per-eye outputs are then combined across both eyes into a patient-level severity estimate. In the paper’s notation,
\[
\hat S =
\begin{cases}
5, & \hat L_L = 1 \text{ or } \hat L_R = 1 \\
\hat D_L + \hat P_L + \hat D_R + \hat P_R, & \text{otherwise}
\end{cases}
\]
where \(\hat D_e = \mathbf{1}\{\hat d_e = \text{large}\}\), \(\hat P_e = \hat p_e\), and \(\hat L_e = \hat l_e\) [1811.07492].

The study compared three training strategies. The best-performing method was Fine-tuned DeepSeeNet, which started with ImageNet-pretrained Inception-v3 and fine-tuned all layers on AREDS images. A second approach, described as an MLP strategy or fixed feature extractor, froze most layers of ImageNet-pretrained Inception-v3 and trained only the last two dense layers of 256 and 128 units. A third approach, Fully-trained DeepSeeNet, started from randomly initialized Inception-v3 and trained all layers on AREDS only [1811.07492]. The reported interpretation was that ImageNet pretraining provided useful low-level initialization, but full fine-tuning was preferable to using Inception merely as a frozen feature extractor because retinal images differ substantially from natural images [1811.07492].

Training details were explicit on several points: Keras with TensorFlow backend, Adam optimizer, learning rate \(\eta = 10^{-4}\), mini-batch size 32, input resolution \(224 \times 224\), and training stopped after 5 epochs when accuracy no longer increased or began to decrease [1811.07492]. Hardware consisted of 32 Intel Xeon CPUs, an NVIDIA GeForce GTX 1080 Ti with 11 GB, and 512 GB RAM [1811.07492]. The paper does not explicitly name the loss function; standard cross-entropy objectives are plausible, but that is not printed as a formal methodological statement in the source text [1811.07492].

## 4. Evaluation results and comparative performance

Evaluation used overall accuracy, sensitivity, specificity, Cohen’s kappa, and ROC curves/AUC [1811.07492]. Confidence intervals were obtained by sampling 450 patients with replacement from the test set, evaluating performance, repeating 2,000 times, and reporting 95% confidence intervals [1811.07492]. The comparison benchmark was unusually strong: 88 retinal specialists who had served as AREDS investigators independently assessed the 450 baseline participants as part of a qualification survey, and those clinical assessments were converted into the same patient-level AREDS Simplified Severity Scale for comparison against Reading Center grades [1811.07492].

The best-performing strategy was Fine-tuned DeepSeeNet. Its patient-based AMD severity classification performance on the 450-patient test set was accuracy 0.671, sensitivity 0.590, specificity 0.930, and Cohen’s kappa 0.558, compared with retinal specialists at accuracy 0.599, sensitivity 0.512, specificity 0.916, and Cohen’s kappa 0.467 [1811.07492]. The alternative strategies performed worse: the MLP strategy achieved accuracy 0.436 and kappa 0.163, while Fully-trained DeepSeeNet achieved accuracy 0.624 and kappa 0.487 [1811.07492].

The lesion-specific subnetworks showed strong discriminative performance. D-Net for large drusen detection achieved accuracy 0.742, sensitivity 0.718, specificity 0.871, kappa 0.601, and AUC 0.94; P-Net for pigmentary abnormalities achieved accuracy 0.890, sensitivity 0.732, specificity 0.957, kappa 0.723, and AUC 0.93; LA-Net for late AMD achieved accuracy 0.967, sensitivity 0.627, specificity 0.987, kappa 0.663, and AUC 0.97 [1811.07492]. The paper’s interpretation is asymmetrical: DeepSeeNet was better than specialists for large drusen and pigmentary abnormalities, but for late AMD it had very high specificity and high AUC while still showing lower sensitivity and lower kappa than specialists [1811.07492].

Class-wise patient-level performance clarifies the same pattern. DeepSeeNet exceeded specialists for true scores 0 through 4, with class accuracies of 90.8%, 43.0%, 48.2%, 34.8%, and 72.7%, compared with specialists at 83.8%, 29.1%, 39.3%, 26.1%, and 45.5%, respectively. For score 5, however, specialists were better at 82.4% versus DeepSeeNet’s 64.7% [1811.07492]. The paper notes from confusion matrices that, for true scores 0 to 4, DeepSeeNet’s errors were usually off by only one scale step [1811.07492].

A later paper built using the DeepSeeNet methods extended this framework to binary lesion detection for geographic atrophy and central geographic atrophy from AREDS color fundus photographs [1906.03153]. That study used 59,812 color fundus photographs from 4,582 participants, trained three binary classifiers with Inception-v3 and ImageNet pretraining, and reported AUCs of 0.933–0.976 for GA detection, 0.939–0.976 for CGA detection from all eyes, and 0.827–0.888 for centrality detection from GA eyes [1906.03153]. The paper states that these models were built using the methods described in DeepSeeNet and that the code and pretrained models were publicly available in the DeepSeeNet repository [1906.03153]. This places the GA/CGA work within a DeepSeeNet-family framework rather than as an independent unrelated system.

## 5. Interpretability, error modes, and limitations

Interpretability in DeepSeeNet is primarily structural. Because the model first predicts drusen category, pigmentary abnormality, and late AMD for each eye before computing the patient score, a clinician can inspect which lesions in which eye drove the final severity output [1811.07492]. This is a stronger form of semantic decomposition than direct image-to-score classification.

The original paper also used t-distributed stochastic neighbor embedding on the 128-dimensional vector from the last dense layer. It reported that, for drusen, small/none and large drusen classes were separated while medium drusen lay between them and showed more overlap; for pigmentary abnormalities and late AMD, positive and negative classes were clearly separated [1811.07492]. This supports the view that the learned representation tracks clinically meaningful lesion structure, and it also helps explain why intermediate drusen categories are difficult.

Image-specific class saliency maps were generated to highlight pixels contributing most to the predictions. The reported finding was that, for drusen, highlighted regions corresponded to visible drusen; for pigmentary abnormality, they aligned with relevant pigment changes; and for late AMD, highlighted areas corresponded to visible advanced lesions [1811.07492]. The paper treats this as evidence that the model was largely attending to clinically meaningful image regions, while also noting that saliency maps do not fully explain internal decision logic.

The qualitative error analysis identifies several recurrent failure modes. Among participants incorrectly classified as having late AMD, 50% had noncentral GA in at least one eye [1811.07492]. This matters because, in the study’s labeling framework, noncentral GA was not counted as late AMD even though it is visually and biologically close to that category. Of the participants incorrectly classified as late AMD, 25% had digital artifacts obscuring the fovea; pale retinas, brightness issues, and digital artifacts could also cause images to be misclassified as geographic atrophy [1811.07492]. The paper therefore explicitly ties some errors to image quality and suggests future quality filtering or corrective preprocessing.

The limitations emphasized in the original study are class imbalance, especially for late AMD; reliance on a single modality, color fundus photographs, without OCT or fundus autofluorescence; sensitivity to image quality; intentionally limited preprocessing; and the need for external validation beyond AREDS [1811.07492]. The study also notes that AREDS began in 1992, so the image acquisition era differs from modern digital retinal imaging [1811.07492]. A plausible implication is that DeepSeeNet combines strong internal validity within a carefully graded clinical-trial image archive with uncertain transfer properties across newer devices, broader pathology mixes, and screening-style acquisition settings.

## 6. Successors, methodological relatives, and application-level reuse

DeepSeeNet generated a distinct methodological lineage. One direct successor was a multi-task deep learning model for automated classification of AMD on the AREDS 9-step non-advanced severity scale [1812.00422]. That work was explicitly built on previous DeepSeeNet and replaced separate single-task submodels with a shared multi-task architecture predicting four AMD characteristics in parallel: drusen area, geographic atrophy, increased pigment, and depigmentation [1812.00422]. The model used 10 Inception-V3 blocks as a shared feature extractor, followed by Global Average Pooling, a dense layer with 1024 units, dropout, and task-specific heads composed of dense layers with 256 and 128 units plus softmax output [1812.00422]. Its principal claim was that joint learning of related AMD characteristics improved performance and robustness, especially on AREDS2 as an independent test set [1812.00422].

The relationship among the main DeepSeeNet-related papers can be summarized concisely.

| Paper | Relation to DeepSeeNet | Primary task |
|---|---|---|
| "DeepSeeNet: A deep learning model for automated classification of patient-based age-related macular degeneration severity from color fundus photographs" [1811.07492] | Canonical DeepSeeNet paper | Patient-based AMD severity classification on the AREDS Simplified Severity Scale |
| "A multi-task deep learning model for the classification of Age-related Macular Degeneration" [1812.00422] | Built on previous DeepSeeNet | AREDS 9-step non-advanced AMD grading via multi-task learning |
| "A deep learning approach for automated detection of geographic atrophy from color fundus photographs" [1906.03153] | Built using DeepSeeNet methods; code in DeepSeeNet repository | GA and CGA detection from color fundus photographs |
| "Development of a Mobile Application for at-Home Analysis of Retinal Fundus Images" [2509.16814] | Application-level integration of adopted DeepSeeNet outputs | Longitudinal monitoring of retinal metrics in a mobile app |

Application-level reuse requires caution. In the 2025 mobile monitoring paper, DeepSeeNet appears as an adopted AMD-analysis component inside a broader retinal fundus image monitoring platform rather than as the object of methodological development [2509.16814]. The paper states that “The DeepSeeNet model was used to output information related to AMD” and attributes to it an accuracy of 0.671, sensitivity of 0.590, and specificity of 0.930, with retinal specialists cited as having accuracy 0.599, sensitivity 0.512, and specificity 0.916; those are prior-model performance characteristics, not outcomes of a new DeepSeeNet experiment in that study [2509.16814]. The meaningful DeepSeeNet-derived outputs presented in the app are Age-Related Macular Degeneration Grade, Drusen Score, Pigmentary abnormalities, Geographic atrophy, and Central Geographic Atrophy [2509.16814].

That same mobile paper contains an internal inconsistency: in the literature review, DeepSeeNet is correctly described as an AMD model that classifies fundus images with the AREDS Simplified Severity Scale, but later one sentence refers to “a model to detect glaucoma” from DeepSeeNet [2509.16814]. The surrounding output descriptions remain AMD-related rather than glaucoma-related, and the paper does not define a coherent DeepSeeNet glaucoma score or provide glaucoma-specific validation [2509.16814]. A careful technical reading therefore indicates that DeepSeeNet’s role in that system is AMD monitoring, not glaucoma detection.

Taken together, these later papers show that DeepSeeNet is both a specific model and an architectural template. In its canonical form, it is an Inception-v3-based, transfer-learning system for bilateral, patient-level AMD severity grading under the AREDS Simplified Severity Scale [1811.07492]. In the surrounding literature, it also functions as a reusable framework for more specialized AMD-related tasks such as GA/CGA detection [1906.03153], as a methodological reference point for multi-task AMD grading systems [1812.00422], and as an adopted component in longitudinal retinal monitoring applications that present AMD-related metrics over time without claiming standalone diagnosis from a single at-home image [2509.16814].

Source: https://www.emergentmind.com/topics/deepseenet