ODIR: Ocular Disease Recognition Dataset

• ODIR is a benchmark dataset of paired retinal fundus images that provides patient-level multi-label annotations across eight ocular disease categories under realistic screening conditions.
• It comprises 10,000 images from 5,000 patients, including demographic and diagnostic metadata, which facilitates research on imbalance handling and multi-label classification.
• Recent studies leveraging ODIR have advanced methods in attention mechanisms, discriminative feature fusion, and binocular modeling to improve ocular disease recognition.

Searching arXiv for the cited ODIR papers and closely related follow-up work. Ocular Disease Intelligent Recognition, commonly cited as ODIR or ODIR-5K, is a benchmark dataset for automated recognition of ocular disease from retinal fundus photographs. It was introduced to reflect routine screening conditions in which ophthalmologists assess both eyes jointly and diagnose multiple diseases within the same patient. The benchmark paper describes 5,000 patients and 10,000 fundus images organized as binocular pairs, with patient-level multi-label annotations over eight disease categories and eye-specific diagnostic keywords for finer-grained analysis. Subsequent studies have reused the dataset under both binocular and per-image formulations, making it a central reference point for multi-label fundus classification, imbalance handling, and binocular feature fusion in ophthalmic machine learning (Li et al., 2021, Bhati et al., 2022).

1. Dataset identity and clinical scope

ODIR was designed for real-world fundus screening rather than single-disease recognition. Its stated goal is multi-disease, binocular fundus screening under realistic acquisition variability, with co-morbidities allowed. The dataset covers eight categories: Normal, Diabetic Retinopathy, Glaucoma, Cataract, Age-related Macular Degeneration, Hypertension-related changes, Pathological Myopia, and Others. In the benchmark formulation, each patient is assigned an 8-dimensional multi-label vector, and both eyes are considered jointly to produce a single patient diagnosis (Li et al., 2021).

The benchmark paper situates ODIR within a large clinical acquisition pipeline. It states that the dataset was drawn from a private clinical repository of more than 1.6 million fundus images collected from 487 hospitals across 26 provinces in China, with representative samples selected after filtering duplicates and low-quality images. Cameras and resolutions vary widely across institutions, and images exhibit variability in color, illumination, and field of view. Age, gender, and diagnostic keywords are included as metadata. Public access is reported through the ODIR2019 Grand Challenge and a GitHub repository, while a later binocular-classification paper cites a Heywhale page and states that the dataset contains anonymized clinical data; licensing terms are not specified in these papers (Li et al., 2021, Huo et al., 25 Apr 2025).

A persistent naming issue appears in the literature. The original benchmark uses OIA-ODIR and ODIR, whereas later work usually uses ODIR-5K. This suggests that the field treats these names as referring to the same benchmark family, while differing papers emphasize either the patient-level binocular release or a derived image-level training view (Li et al., 2021, Bhati et al., 2022).

2. Structure, labels, and unit of supervision

The defining structural property of ODIR is binocular pairing. Each patient record includes left and right color fundus photographs, demographic information, and physicians’ diagnostic keywords. The benchmark paper specifies patient-level multi-label diagnosis, while later work emphasizes that the released dataset provides “a common target label for a pair of fundus image.” This distinction is consequential: the natural unit of annotation is the eye pair, but many CNN pipelines operate on single images (Li et al., 2021, Bhati et al., 2022).

The eight disease categories are stable across the literature, although abbreviations vary. The benchmark paper uses $N$, $D$, $G$, $C$, $A$, $H$, $M$, and $O$, corresponding to Normal, Diabetic retinopathy, Glaucoma, Cataract, Age-related macular degeneration, Hypertension-related changes, Pathological myopia, and Others. A later binocular model uses the abbreviations NL, DR, GL, CT, AMD, HR, MY, and OTH. The semantic content is consistent, but the notation changes across papers (Li et al., 2021, Huo et al., 25 Apr 2025).

A major reuse strategy is per-eye label derivation from diagnostic keywords. In the DKCNet study, the authors split the common pair-level target into per-image labels using the diagnostic descriptors for the left and right eyes, then train on left and right fundus images individually. That pipeline also removes images marked with artifacts such as “low-quality image,” “optical disk photographically invisible,” “lens dust,” and “pimage offset” to reduce false recognition. By contrast, DMS-Net keeps the binocular pairing explicit and ingests paired images through a weight-shared Siamese backbone (Bhati et al., 2022, Huo et al., 25 Apr 2025).

Two misconceptions recur in secondary discussion. First, ODIR is not inherently a monocular dataset; its native organization is binocular. Second, in DMS-Net the term “dual-modal” does not denote an additional imaging modality such as OCT. The paper explicitly states that “dual-modal” refers to combining global contextual features from binocular images with local edge-sensitive features learned within the network (Huo et al., 25 Apr 2025).

3. Class distribution, imbalance, and preprocessing conventions

ODIR is strongly imbalanced. At the patient level, the benchmark reports the following all-case label counts, with sums exceeding the number of patients because co-occurrence is allowed: Normal 1624, Diabetic retinopathy 1620, Glaucoma 305, Cataract 308, Age-related macular degeneration 238, Hypertension-related changes 149, Pathological myopia 243, and Others 1393. The benchmark explicitly notes that Hypertension-related changes has fewer than one-tenth the positive patients of Normal, and that the Others class is both large and diverse (Li et al., 2021).

Later per-image reformulations retain the imbalance while changing the counts. After separating left and right eye images and assigning per-image labels from diagnostic keywords, the DKCNet paper reports the following image-level distribution:

Category	Patient-level count	Per-image count used by DKCNet
Normal	1624	1135
Diabetes / Diabetic Retinopathy	1620	1131
Glaucoma	305	207
Cataract	308	211
AMD	238	171
Hypertension	149	94
Myopia	243	177
Others	1393	944

The Others category is especially problematic. The DKCNet paper describes it as lesions related to 12 different ophthalmic diseases, which makes feature learning difficult and creates label-granularity issues. This heterogeneity is one of the central reasons later methods emphasize discriminative region modeling, channel recalibration, or binocular correspondence rather than relying on deeper backbones alone (Bhati et al., 2022).

Imbalance handling differs substantially across studies. The DKCNet work constructs oversampled and undersampled variants using random sampling and a class balancing factor. Minority classes are expanded according to $M_{minor} = N_{minor} \times (1 + k)$, and oversampling uses augmentations including flip; rescaling with ratios 0.5, 0.7, 0.8, 0.9; crop; rotation; contrast change; hue, saturation, and gamma changes. By contrast, DMS-Net states that it preserves class distribution during preprocessing in order to “moderately maintain the inherent class imbalance,” and combines that strategy with non-uniform illumination correction and CutMix (Bhati et al., 2022, Huo et al., 25 Apr 2025).

Preprocessing is likewise non-uniform across papers. The DKCNet study crops to a square field of view by detecting the first non-black pixel to define the mask border, then resizes to $224 \times 224$. The benchmark paper recommends fundus cropping and illumination or color normalization, but does not define a canonical preprocessing stack. DMS-Net reports non-uniform illumination correction but does not specify resizing, cropping, normalization, or artifact removal beyond that step (Li et al., 2021, Bhati et al., 2022, Huo et al., 25 Apr 2025).

4. Benchmark protocol and evaluation practice

The original benchmark defines the primary task as multi-label patient-level classification over $D$0 diseases, using both eyes as input. Each patient has a binary vector $D$1, and models output sigmoid-activated probabilities. The benchmark reports Cohen’s Kappa, F1-score, AUC, and a “Final-score,” defined as the mean of Kappa, F1, and AUC. Outputs are per-class probabilities, and the paper notes that practitioners may threshold at 0.5 or tune thresholds on the off-site set, but the exact thresholding strategy used for the published results is not specified beyond the use of scikit-learn (Li et al., 2021).

The canonical split is patient-level and preserves binocular pairing: 3,500 patients for training, 500 for off-site test, and 1,000 for on-site test. The split is intended to probe generalization under distribution shift between off-site and on-site conditions. Under this benchmark, Inception-v4 with element-wise multiplication achieved the strongest off-site Final-score among the evaluated entries, with Kappa $D$2, F1 $D$3, AUC $D$4, and Final $D$5. On the on-site test, Vgg-16 with element-wise multiplication reached Kappa $D$6, F1 $D$7, AUC $D$8, and Final $D$9 (Li et al., 2021).

Subsequent reuse papers diverge from this protocol. The DKCNet study treats the problem as multi-class, multi-label classification with binary cross-entropy, trains with SGD using initial learning rate 0.0005, decay $G$0, batch size 16, and 100 epochs on an NVIDIA T4 GPU, and assigns labels with confidence greater than 0.5 as positive. It reports an 80% training and 20% validation split and states that “results for 10-fold cross validation are shown.” DMS-Net, by contrast, reports ODIR-5K results with Accuracy, Recall, Precision, F1, AUC, and Cohen’s kappa, but does not specify the train/validation/test split and does not describe the loss function explicitly (Bhati et al., 2022, Huo et al., 25 Apr 2025).

This protocol heterogeneity matters for interpretation. Direct numerical comparison across the benchmark paper, DKCNet, and DMS-Net is limited because the unit of prediction, split definition, sampling regime, and sometimes even the loss function differ across studies. A plausible implication is that ODIR functions simultaneously as a benchmark dataset and as a flexible substrate for alternative formulations of binocular or per-eye disease recognition (Li et al., 2021, Bhati et al., 2022, Huo et al., 25 Apr 2025).

5. Methodological development on ODIR

The original benchmark’s central methodological conclusion is that simply increasing network scale does not yield reliable gains for multi-disease recognition. Nine ImageNet-pretrained CNNs were adapted for binocular multi-label classification with late feature fusion. Three fusion strategies were tested: element-wise sum, element-wise multiplication, and concatenation followed by a fully connected layer. Sum and multiplication consistently outperformed concatenation, and deeper networks did not necessarily outperform shallower ones. The paper therefore argues that structured feature fusion is needed to combine characteristics of multiple diseases and binocular relationships (Li et al., 2021).

The DKCNet study is an example of this shift toward structured discriminative modeling. DKCNet attaches an attention block and a squeeze-and-excitation block to a CNN backbone. The architecture uses dilated convolutions with dilation rates 2, 3, and 4 on $G$1 kernels, global max and average pooling, channel shuffle, and channel-wise recalibration to capture multi-scale, region-wise lesion features at low additional cost. Across ResNet-101, InceptionV3, and InceptionResNet backbones, the best reported ODIR-5K performance used InceptionResNet and reached AUC 96.08, F1-score 94.28, and Cohen’s kappa 0.81. The same paper reports external evaluation on unseen datasets, with AUC 89.37 and F1-score 87.75 on Messidor, AUC 93.14 and F1-score 91.42 on G1020, and AUC 94.18 and F1-score 91.15 on the Joint Shantou International Eye Centre dataset (Bhati et al., 2022).

DMS-Net instead exploits the binocular structure directly. It uses weight-shared Siamese ResNet-152 backbones, a Multi-Scale Context-Aware Module with adaptive average pooling at $G$2 and $G$3, global max and average pooling, and spatial attention, and a Dual-Modal Feature Fusion module with bidirectional attention and adaptive residual fusion. It also introduces difference and sum branches to model left-right asymmetry and aggregated edge-like information. On ODIR-5K, the full model reports Accuracy 80.5%, Recall 86.1%, Cohen’s kappa 83.8%, Precision 0.872, F1 score 0.863, and AUC 0.972. Ablation tables in the same paper show consistent drops when removing the sum branch, difference branch, DMFF, or MSCAM, and ResNet-152 performs best among the tested backbones (Huo et al., 25 Apr 2025).

Taken together, these lines of work demonstrate two dominant interpretations of ODIR. One treats it as a per-image multi-label recognition problem that requires explicit handling of label noise and class imbalance; the other treats it as a binocular correlation problem in which symmetric and asymmetric pathology across eyes carries diagnostic signal. Both interpretations are directly supported by the dataset’s design and annotation structure (Bhati et al., 2022, Huo et al., 25 Apr 2025).

6. Limitations, reproducibility issues, and place in the literature

Several constraints of ODIR are consistently emphasized. Class imbalance is severe and long-tailed. The Others category is coarse and heterogeneous. Device variability, illumination diversity, and resolution changes introduce domain shift. Common artifacts include lens stains, glare or highlights, and variable exposure. These properties make ODIR clinically realistic, but they also complicate benchmarking and can inflate the effect of preprocessing choices (Li et al., 2021, Bhati et al., 2022).

Reproducibility is uneven across papers. The benchmark paper provides patient-level splits and a public-access pathway, but not detailed training hyperparameters such as normalization, augmentation schedule, optimizer settings, or batch size. DKCNet specifies optimization and preprocessing more fully, but the AUC aggregation method is not reported. DMS-Net presents extensive ablations, yet does not specify the train/validation/test split or the loss function. Access information is also fragmented: the benchmark points to GitHub and the challenge site, later work points to Heywhale, and licensing terms are not reported in the cited papers (Li et al., 2021, Bhati et al., 2022, Huo et al., 25 Apr 2025).

ODIR is also important as a comparative reference point. A later fundus foundation-model paper contrasts it with larger private corpora and notes that ODIR covers multiple diseases but is smaller than very large pretraining resources. This suggests that ODIR’s main role in the literature is not large-scale representation learning from raw volume alone, but downstream evaluation of multi-disease recognition under binocular, imbalanced, and heterogeneous clinical conditions (Jang et al., 2024).

Within the broader landscape of ophthalmic datasets, ODIR is distinctive because it combines moderate scale, binocular organization, multi-label disease annotation, diverse acquisition devices, and auxiliary metadata such as age, gender, and diagnostic keywords. Its continuing reuse in per-eye classification, binocular Siamese modeling, and generalization studies indicates that it has become a canonical benchmark for testing how retinal AI systems handle co-morbidity, pairwise structure, and acquisition variability in fundus screening (Li et al., 2021, Huo et al., 25 Apr 2025).