XDR-LVLM: Explainable DR Diagnosis

• XDR-LVLM is an explainable framework for diabetic retinopathy diagnosis that combines severity grading with lesion-grounded natural language reports.
• It integrates a specialized medical visual encoder, LVLM core module, and vision-language connector to align retinal features with clinical explanations.
• Multi-task prompt engineering and multi-stage fine-tuning enable the model to achieve high diagnostic accuracy and robust explanation quality.

Searching arXiv for the cited topic and closely related work to ground the article. XDR-LVLM is an explainable diabetic retinopathy diagnosis framework built around a Vision-Language Large Model (LVLM) for fundus-image analysis. It is presented as a system that combines high-precision diabetic retinopathy severity grading with natural-language explanation generation, so that the output is not only a disease label but also a report that explicitly includes severity grading, key pathological concepts, and the diagnostic basis linking observed retinal findings to the prediction (Ito et al., 21 Aug 2025).

1. Clinical scope and design objective

XDR-LVLM is defined around automated diagnosis of diabetic retinopathy from retinal fundus photographs, with emphasis on two coupled tasks: severity grading and explanation. The clinical motivation is that diabetic retinopathy screening depends not only on recognizing whether disease is present, but on identifying lesion patterns such as hemorrhages, exudates, and microaneurysms, and relating those findings to severity. The paper positions conventional high-performing CNN and ViT classifiers as black-box models whose outputs are difficult to audit clinically, while post hoc visual explanations such as heatmaps remain indirect and do not explicitly name pathological findings or articulate the diagnostic rationale (Ito et al., 21 Aug 2025).

The target output therefore extends beyond classification. XDR-LVLM is designed to produce a report containing three content layers: severity diagnosis, pathological concept identification, and a textual explanation of why those findings support the diagnosis. The severity space is the standard five-class diabetic retinopathy scale: No DR, Mild DR, Moderate DR, Severe DR, and Proliferative DR. The pathological concepts named in the paper include hemorrhages, exudates, microaneurysms, cotton wool spots, and neovascularization; the detailed per-concept evaluation further reports Microaneurysms, Hemorrhages, Hard Exudates, Soft Exudates (Cotton Wool Spots), Neovascularization, and Intraretinal Microvascular Abnormalities (IRMA) (Ito et al., 21 Aug 2025).

A common misconception in this area is that explainability necessarily reduces diagnostic performance. The reported results are presented as the opposite: XDR-LVLM is described as an explainable multimodal diagnostic system whose disease-diagnosis and concept-detection metrics slightly exceed the strongest reported black-box or concept-based baselines on the DDR benchmark. This suggests that, at least in the paper’s experimental setting, explicit report generation and lesion-grounded reasoning are not treated as a post hoc add-on but as part of the core diagnostic formulation.

2. Model architecture and information flow

The framework is organized around three named components: a Medical Vision Encoder, an LVLM Core Module, and a vision-language connector that projects image features into the LLM space. The Medical Vision Encoder is described as a Vision Transformer pre-trained on large medical image corpora, including ophthalmic data, as well as ImageNet. Given a fundus image $I$, the encoder produces visual features

$F_V = E_V(I).$

The LVLM Core Module is described as being based on architectures such as LLaVA, Flamingo, or Llama-series integration; in the implementation summary, the system is instantiated as a LLaVA variant with a ViT visual encoder and a Llama2-series language decoder. Between the vision and language components, the connector—described as an MLP or Q-Former—maps visual features into the decoder’s embedding space, producing projected visual features $F'_V$. These projected features are concatenated with the task prompt $P_T$ to form the multimodal input sequence

$S_{input} = [F'_V; P_T].$

The report-generation function is then given as

$T_{report} = L(F'_V, P_T),$

where $L$ denotes the LVLM core. In prose, the data flow is: fundus image $\rightarrow$ medical visual features $\rightarrow$ connector projection $\rightarrow$ concatenation with task prompt $F_V = E_V(I).$0 autoregressive report generation. The report is intended to state the DR grade, identify lesions, and explain how those findings support the grade (Ito et al., 21 Aug 2025).

This design gives XDR-LVLM a concept-grounded character without formalizing a strict concept bottleneck. Pathological concepts are neither merely auxiliary labels nor pure post hoc explanations; they function as clinically meaningful intermediate content in the generated report. A plausible implication is that the model is meant to align multimodal representations with the structure of ophthalmic reasoning rather than only optimize for end-class labels.

3. Prompting strategy and multi-stage fine-tuning

A central mechanism in XDR-LVLM is Multi-task Prompt Engineering. Rather than using a single generic instruction, the paper defines separate prompts for diagnosis and concept identification. The diagnosis prompt is:

“Please analyze this fundus image and determine the severity level of Diabetic Retinopathy. Provide a detailed explanation for your diagnosis.”

The concept prompt is:

“Identify and describe all Diabetic Retinopathy-related pathological concepts present in this image, such as hemorrhages, exudates, microaneurysms, cotton wool spots, or neovascularization. Point out their locations and features.”

These prompts are used to route the same LVLM toward different subtasks: severity grading with rationale, or lesion/concept identification with descriptive detail. The paper does not present a strict report template or a chain-of-thought supervision protocol, but it does treat prompt-conditioned generation as the primary control interface for multitask behavior (Ito et al., 21 Aug 2025).

Training is described as Multi-stage Fine-tuning. In Stage 1, visual-language alignment is performed on a diverse set of image-text pairs using image captions and descriptive tags. The paper presents the alignment objective as

$F_V = E_V(I).$1

where $F_V = E_V(I).$2 is the connector and $F_V = E_V(I).$3 is the text embedding function. In Stage 2, the aligned multimodal model is instruction-tuned on the DDR dataset using prompt-output pairs, with autoregressive generation loss

$F_V = E_V(I).$4

The implementation summary specifies AdamW, a cosine annealing learning-rate schedule, and NVIDIA A100 GPUs. At the same time, several details remain underspecified. The paper does not provide learning-rate values, batch size, epoch counts, exact freezing or joint-tuning policy, or named Stage 1 alignment corpora. It also states that Stage 2 uses “ground-truth diagnostic and explanatory texts,” but does not explain how those texts were obtained from DDR, which is otherwise described as a dataset with severity labels and lesion bounding boxes rather than native report text. That omission is material because it affects reproducibility and the interpretation of the explanation-generation supervision.

4. Dataset, tasks, and evaluation protocol

All main experiments are reported on the DDR dataset, which the paper describes as containing fundus images, diabetic retinopathy severity labels from No DR to Proliferative DR, and bounding box annotations for pathological concepts. The standard training, validation, and testing splits provided with DDR are said to be used, although exact split sizes are not reported (Ito et al., 21 Aug 2025).

The evaluation protocol covers three targets. The first is disease diagnosis, measured with Balanced Accuracy (BACC) and F1 for five-class DR severity grading. The second is concept detection, also evaluated with BACC and F1. The third is explanation quality, assessed by human raters rather than automatic language-generation metrics. The human study uses 100 randomly selected fundus images and five reviewers—three experienced ophthalmologists and two medical residents—who rate outputs for Fluency, Accuracy of Explanation, and Clinical Utility.

The diagnosis baselines reported are CBM, CLAT, Black-box (ViT Base), and Black-box (Task-Specific). For concept detection, the reported baselines are CBM and CLAT. The comparison is therefore primarily against DR-specific classifiers and concept-based models rather than against other medical report-generating LVLMs. This matters when interpreting the results: the paper demonstrates superiority within that comparison set, but does not directly benchmark against a generic medical LVLM baseline such as a vanilla LLaVA-style fundus-report generator.

5. Reported performance

The headline numbers are concise. XDR-LVLM reports 84.55% BACC and 79.92% F1 for disease diagnosis, and 77.95% BACC and 66.88% F1 for concept detection. Human evaluation reports 93.1% Fluency, 88.7% Accuracy of Explanation, and 82.9% Clinical Utility (Ito et al., 21 Aug 2025).

Evaluation target	Metric	XDR-LVLM
Disease diagnosis	BACC / F1	84.55% / 79.92%
Concept detection	BACC / F1	77.95% / 66.88%
Human evaluation	Fluency / Accuracy of Explanation / Clinical Utility	93.1% / 88.7% / 82.9%

Against reported baselines, disease-diagnosis BACC exceeds Black-box (Task-Specific) at 83.38 and CLAT at 72.81, while F1 slightly exceeds CLAT at 78.87. For concept detection, XDR-LVLM exceeds CLAT from 76.64 BACC / 64.53 F1 to 77.95 BACC / 66.88 F1. The paper therefore presents the system as one in which explainability is not traded off against standard predictive metrics.

Classwise diagnosis results show the strongest F1 scores for No DR (91.3), Proliferative DR (86.1), and Moderate DR (84.1), with the weakest class being Severe DR (69.7). For lesion concepts, the best reported category is Hard Exudates at 88.0 BACC / 85.5 F1, followed by Hemorrhages and Neovascularization, while Soft Exudates (Cotton Wool Spots) and IRMA are weaker. This pattern is clinically plausible in the narrow sense that subtler or less visually distinct findings are harder for the model, though the paper does not formalize lesion-difficulty analysis beyond the per-concept table.

The ablation study isolates three design choices. Replacing the medical visual encoder with a generic ImageNet-pretrained ViT reduces disease diagnosis to 81.23 BACC / 76.51 F1 and concept detection to 71.88 BACC / 61.05 F1. Removing Multi-task Prompt Engineering yields 82.91 / 77.85 and 75.32 / 64.20. Replacing Multi-stage Fine-tuning with single-stage end-to-end fine-tuning yields 82.07 / 77.10 and 73.96 / 62.91. The largest drop comes from removing the medical visual encoder, indicating that retinal-domain visual specialization is the most consequential component in the reported setup (Ito et al., 21 Aug 2025).

6. Explainability claims, failure modes, and limitations

The explanatory output is one of XDR-LVLM’s defining features. The paper describes typical report patterns such as: “The image shows numerous large hemorrhages and extensive neovascularization, indicative of severe proliferative DR,” “Presence of a few microaneurysms and small dot hemorrhages suggests mild non-proliferative DR, warranting close monitoring,” and “No signs of diabetic retinopathy such as hemorrhages, exudates, or neovascularization are observed.” These examples illustrate the intended style: lesion naming, severity linkage, and clinically legible prose (Ito et al., 21 Aug 2025).

At the same time, the paper is explicit that explanation quality is not identical to explanation faithfulness. In misclassified cases, the generated explanation is often internally consistent with the model’s mistaken prediction. That means the output may be a coherent rationale for the model’s belief state rather than a guaranteed faithful account of the true image evidence. This is an important limitation for any encyclopedia treatment of XDR-LVLM: the system is more interpretable than a pure black-box classifier, but the deeper faithfulness problem remains unresolved.

Several other limitations are either stated directly or implied by omissions. The paper does not specify the provenance of the “ground-truth diagnostic and explanatory texts” used for instruction tuning on DDR. Exact optimization hyperparameters, preprocessing, augmentation, and freezing policies are not reported. Evaluation is confined to DDR, so generalization across institutions, devices, or retinal datasets is untested. The model is weaker on subtle lesions such as Soft Exudates and IRMA. It does not provide uncertainty quantification, calibration analysis, or prospective clinical validation. The conclusion proposes future work on subtle-lesion detection, uncertainty-aware explanations, other retinal diseases, multimodal extension with OCT or patient history, and prospective real-world evaluation.

Taken together, XDR-LVLM is best understood as a specialized explainable LVLM for diabetic retinopathy in which a medical visual encoder, prompt-conditioned report generation, and staged multimodal fine-tuning are used to couple severity grading with lesion-grounded explanation. Its reported contribution lies less in introducing a new general LVLM architecture than in demonstrating a domain-specific configuration where diagnostic prediction and natural-language clinical rationale are trained and evaluated as a single ophthalmic system (Ito et al., 21 Aug 2025).