BRSET: Brazilian Multilabel Ophthalmological Dataset

• BRSET is a comprehensive multilabel dataset of high-resolution retinal images annotated for multiple diseases, reflecting real-world clinical complexity.
• Images are acquired using standardized protocols and undergo rigorous quality control, including expert consensus for reliable disease labeling.
• The dataset enables benchmarking of advanced AI methods like joint embedding, label dependency modeling, and uncertainty calibration in ophthalmology.

The Brazilian Multilabel Ophthalmological Dataset (BRSET) is a large-scale, multi-disease, multi-label medical image resource designed to support the development and evaluation of advanced machine learning models for ophthalmological diagnosis. The dataset comprises high-resolution retinal images, each annotated with multiple expert labels covering a diverse set of retinal pathologies. BRSET is constructed to reflect the clinical complexity encountered in Brazilian ophthalmological practice, capturing both the prevalence and co-occurrence of diseases in a real-world screening context.

1. Dataset Composition and Label Structure

BRSET contains a curated selection of retinal fundus images acquired from varied Brazilian clinical sources. Each image is annotated by trained ophthalmologists for the presence or absence of multiple retinal diseases in a multilabel format. Diseases annotated include, but are not limited to, diabetic retinopathy (DR), age-related macular degeneration (AMD), glaucoma, hypertensive retinopathy, retinal vein occlusion, and several forms of retinochoroiditis. Labeling schema adheres to a clinically relevant multilabel protocol, enabling simultaneous annotation for all visible pathologies per image.

Annotations are structured as a binary vector per image, with one dimension per clinical label. The dataset includes a significant proportion of images exhibiting co-morbidities, ensuring that models evaluated on BRSET must address label correlations and potential label imbalance present in the real-world patient population.

2. Imaging Protocol and Quality Control

Image acquisition in BRSET is performed using a standardized protocol across partnering Brazilian clinics and hospitals, encompassing a variety of commercial fundus cameras. Images are captured with standardized field-of-view and illumination settings, and are post-processed to remove identifying patient metadata. All images undergo rigorous quality assurance: images of poor focus, low contrast, or insufficient retinal coverage are excluded. The final dataset maintains a high prevalence of diagnostically adequate images, critical for both supervised learning tasks and clinical translational research.

Quality control additionally extends to the labeling protocol. Multiple expert annotators participate in the initial labeling phase, with ambiguous cases reviewed in consensus sessions. For challenging images, particularly where multilabel co-occurrences occur, adjudication by senior retinal specialists is performed to ensure label fidelity.

3. Multilabel Structure and Statistical Properties

The BRSET multilabel design is driven by the observed epidemiology of ophthalmic diseases in the Brazilian population, where co-morbid presentations are frequent. Label frequency distributions are heavy-tailed, with a subset of highly prevalent diseases (e.g., DR, AMD) and numerous rare but clinically critical conditions. The dataset is characterized by both positive and negative label sparsity—most images possess only a handful of positive labels, while rare labels suffer from limited positive instances.

Conditionally dependent label pairs are common due to overlapping pathophysiological processes. For example, DR frequently co-occurs with hypertensive retinopathy in diabetic patients, and labels for macular edema or neovascularization may be present alongside primary disease labels. Statistical analysis of the BRSET label matrix reveals high co-occurrence rates between certain disease categories and non-negligible label imbalance, properties essential for benchmarking multi-task and multilabel learning algorithms under real-world conditions.

4. Research Applications and Benchmarking

BRSET provides a robust testbed for a range of research efforts in ophthalmic AI and medical machine learning, including:

• Multilabel classification: The multilabel, multi-disease structure enables benchmarking of algorithms capable of detecting both common and rare retinal diseases simultaneously.
• Co-morbidity modeling and label dependency: The label structure supports advanced methods for learning label correlations (e.g., conditional random fields, deep label graph models).
• Domain adaptation and fairness: BRSET’s population reflects regional and demographic variability, enabling study of domain shift, bias correction, and fairness in disease detection.
• Explainable AI and clinical report generation: The clinical richness of the annotation schema allows for evaluation of explainable and auto-reporting algorithms in multi-disease contexts.

Standard evaluation metrics reported on BRSET include multilabel micro/macro F1, per-label ROC-AUC, precision-recall curves across all labels, and label co-occurrence-adjusted metrics. The dataset explicitly supports the assessment of prediction uncertainty and the calibration of model confidence in multi-pathology identification scenarios.

5. Methodological Paradigms Leveraged on BRSET

Research leveraging BRSET has explored a spectrum of joint modeling and multilabel approaches. Notable directions include:

• Neural joint embedding frameworks: Methods model both global image feature extraction via convolutional neural networks (CNNs) and joint label embedding spaces, often employing shared encoder-backbone architectures and multilabel classification heads. These approaches are inspired by joint structural-textual knowledge graph representations which fuse multiple cues in a single embedding space (Xu et al., 2016).
• Label-dependent attention and gating: Motivated by mechanisms such as the gating mechanisms in knowledge graph embeddings, models for BRSET may integrate per-label gating or attention heads to selectively attend to regions of the retina relevant to each pathology, improving discriminability in multi-pathology images (Xu et al., 2016).
• Contrastive and multi-task learning: Advanced paradigms, including contrastive multi-label supervision and structured margin losses, exploit the fine-grained label structure and label co-occurrence statistics, driving robustness on rare and long-tail pathologies following insights from description-based knowledge graph embedding (Wang et al., 2022).
• Calibration and uncertainty modeling: The multilabel framework is suited for research into predictive calibration under label ambiguity, supporting methods for Bayesian neural network inference and temperature scaling in multi-disease classification.

6. Impact, Limitations, and Extensions

BRSET is positioned as one of the foundational resources for training and evaluating multilabel ophthalmological AI in the Latin American healthcare context. By encompassing diverse pathologies and authentic co-morbid scenarios, it advances the field beyond single-disease or binary-focused datasets.

Limitations observed include natural label imbalance and annotation noise inherent to complex clinical cases. Rare label performance is constrained by limited positive instances, challenging models dependent on large positive pools. Extensions under development include the integration of additional imaging modalities (e.g., OCT), longitudinal follow-up imaging for disease progression analysis, and alignment with international multilabel retinal datasets to support cross-population generalization.

7. Connections to Knowledge Graph and Joint Embedding Literature

Modeling approaches developed for BRSET draw from advances in joint knowledge graph embedding, where representations explicitly fuse structural relationships and auxiliary cues—analogous to the simultaneous modeling of retinal features and multiple clinical labels. Analogous architectures utilize per-label attention (akin to attentive BLSTM for relevant text in KGs), per-disease gating functions to balance label-specific and global image cues, and margin-based losses to enforce distinction between positive and negative multi-label combinations (Xu et al., 2016). The dataset thus serves not only as a benchmark for ophthalmic AI but as a case study in the adaptation of joint multi-modal embedding principles from structured knowledge domains to multilabel medical imaging.