EyePCR-MLLM: Ophthalmic Surgery Cognition

• The paper demonstrates that EyePCR-MLLM is a domain-adapted variant of Qwen2.5-VL-7B, fine-tuned on the EyePCR benchmark to achieve leading MCQ accuracy in fine-grained perceptual tasks.
• It leverages multi-view perceptual supervision and a clinical knowledge graph to bolster structured knowledge comprehension and reasoning across diverse surgical task domains.
• The model excels in perceptual recognition while underperforming in generative reasoning, revealing a gap compared to expert-level surgical cognition essential for clinical applications.

EyePCR-MLLM is a domain-adapted variant of Qwen2.5-VL-7B introduced in conjunction with EyePCR, a large-scale benchmark for ophthalmic surgery analysis designed to evaluate cognition across Perception, Comprehension and Reasoning in surgical settings (Wang et al., 19 Sep 2025). It is fine-tuned on the EyePCR benchmark to improve fine-grained perception, structured knowledge comprehension, and clinically grounded reasoning in ophthalmic surgery. Within the reported experiments, EyePCR-MLLM achieves the highest accuracy on MCQs for Perception among compared models and outperforms open-source models in Comprehension and Reasoning, while rivaling commercial models such as GPT-4.1 (Wang et al., 19 Sep 2025).

1. Model identity and architectural scope

EyePCR-MLLM is explicitly described as a domain-adapted variant of Qwen2.5-VL-7B, with 7 billion parameters inherited from the base model (Wang et al., 19 Sep 2025). The paper does not provide architectural schematics for EyePCR-MLLM, and it does not describe the vision encoder, multimodal connector, or language decoder in detail. By implication, the model inherits Qwen2.5-VL’s standard image/video encoders and an LLM decoder.

The input modalities used throughout the benchmark are image, video, and text. Typical input resolution and context length for training or inference are not specified. The underlying videos were curated to ensure at least 720p quality during preprocessing, but the model’s actual input resolution is not reported.

A central clarificatory point is that the paper does not introduce explicit ophthalmology-specific perception heads, knowledge-graph fusion adapters, or task-specific decoders for EyePCR-MLLM. Domain specialization is instead achieved via instruction-style fine-tuning on EyePCR’s PCR tasks. Likewise, temporal or video-specific modeling details, such as frame sampling strategy or temporal attention, are not reported. This makes EyePCR-MLLM, in the form documented, an adaptation and evaluation vehicle rather than a newly specified multimodal architecture.

2. Benchmark substrate and data regime

EyePCR-MLLM is inseparable from the EyePCR benchmark, since the model’s domain adaptation is defined through fine-tuning on that benchmark’s Perception, Comprehension, and Reasoning hierarchy (Wang et al., 19 Sep 2025). EyePCR contains 213,883 VQAs in total, comprising 64,425 MCQs and 149,458 OEQs. Across the PCR hierarchy, the distribution is Perception 102,827, Comprehension 24,693, and Reasoning 86,363.

The dataset is built from 1,544 curated videos drawn from public ophthalmic repositories and datasets, specifically OphNet, LensID, Cataract-1k, Eyetube, and AAO repositories. At the segmentation level, EyePCR contains 82,893 segments in total, with 77,133 in train and 5,760 in test. The split is performed at patient level to avoid leakage. All videos are publicly sourced and de-identified, although a formal license or download link is not provided in the paper.

The benchmark’s perceptual supervision is unusually granular. It defines 1,048 fine-grained attributes distributed across seven semantic views and three attribute levels. The multi-level breakdown is L1 = 76, L2 = 240, and L3 = 732. The seven semantic views, denoted V1–V7, are Ocular Anatomy, Pathological Diagnosis, Surgical Operation, Device & Instrumentation, Intraoperative Anatomy, Medicine, and Monitoring. This structure is intended to support multi-view perception rather than coarse scene recognition.

3. PCR hierarchy and knowledge-graph grounding

EyePCR organizes evaluation around a three-level cognitive schema: Perception, Comprehension, and Reasoning (Wang et al., 19 Sep 2025). Perception is grounded in the seven semantic views and three attribute levels. Comprehension and Reasoning are supported by a medical knowledge graph and clinically grounded task construction.

The knowledge graph contains 25,567 triplets, 16,034 entities, 3,507 relations, and 11,462 logical paths, of which 9,499 are 2-hop and 1,963 are at least 3-hop. Its formalism is given as $G = (V, E)$ with triplets $E \subseteq V \times R \times V$. The triplets are either entity–action–attribute or entity–relation–entity; the paper gives the example $$[\mathrm{OVD}] \rightarrow \text{injected into} \rightarrow \text{anterior chamber}$$. Reasoning paths are represented as

$$\pi = [(v_0, r_1, v_1), \ldots, (v_{L-1}, r_L, v_L)].$$

These graphs are used to ground comprehension and reasoning questions, but no KG-specific training losses are reported for EyePCR-MLLM. No knowledge-graph module is integrated into the model architecture; the KGs are used on the dataset side to construct grounded VQAs and reasoning paths.

The reasoning component is organized into four clinically grounded task families. D1 Procedural Flow Understanding takes a video segment and a question about step, ordering, or causal flow, and requires a free-text answer explaining sequence or dependencies. D2 Surgical Intent Inference takes context and action, and requires a free-text rationale for the action, aligned with observed anatomy or pathology. D3 Intraoperative Decision Simulation takes an ongoing scene and requires forecasting of the next step or contingency planning under uncertainty. D4 Anomaly/Risk Awareness Assessment takes a scene with deviation and requires identification of potential complications or risk implications.

MCQs and OEQs are automatically generated using Gemini-2.5-Pro from aligned video segments, attributes, subtitles, and the semantic memory graph. Whisper is used for speech-to-text and CnOCR for subtitle extraction to support structured knowledge building and KG construction. Clinical expert screening of videos is documented, but inter-annotator agreement metrics are not reported.

4. Domain adaptation and training formulation

The reported adaptation strategy is fine-tuning Qwen2.5-VL-7B on the EyePCR dataset across the three cognitive levels of Perception, Comprehension, and Reasoning (Wang et al., 19 Sep 2025). The paper emphasizes multi-task exposure and instruction-style VQA training over MCQs and OEQs, but does not publish the training objectives, loss weights, prompt templates, schedule mixing, or curriculum.

Multi-view perception supervision is explicitly tied to the seven semantic views V1–V7 and the three attribute levels L1–L3. This means that the model is not merely exposed to generic ophthalmic imagery; it is exposed to a structured ontology of surgical content spanning anatomy, pathology, actions, devices, medicines, monitoring, and intraoperative anatomy.

The implementation details are notably sparse. Hardware, batch sizes, epochs, learning rates, optimizer, data augmentation, context windows, and inference speeds are not provided. No code or checkpoints are cited. Statistical significance measures and confidence intervals are not reported, and the paper does not include ablation studies. The patient-level split protocol is therefore one of the main reported reproducibility controls.

A frequent misunderstanding would be to treat EyePCR-MLLM as a fully specified ophthalmology-native architecture. The documented evidence does not support that interpretation. The paper reports domain adaptation by instruction-style fine-tuning on a clinically grounded benchmark, but leaves the internal training recipe and architectural modifications largely unspecified.

5. Evaluation protocol and empirical results

Evaluation is divided between MCQs and OEQs (Wang et al., 19 Sep 2025). MCQs are measured by Accuracy. OEQs are measured by BLEU-1–4, ROUGE-1/2/L, and GPT-4.1-based rubric scoring on a 0–5 Likert scale for factuality, specificity, and completeness. Standard definitions associated with the reported metrics are:

$$\mathrm{Accuracy} = \frac{\sum_{i=1}^{N} \mathbf{1}(\hat{y}_i = y_i)}{N}$$

$$\mathrm{BLEU}_n = BP \cdot \exp\!\left(\sum_{k=1}^{n} w_k \cdot \log p_k\right)$$

$$\mathrm{ROUGE\text{-}L} = \frac{(1+\beta^2)\cdot R \cdot P}{R+\beta^2 \cdot P}.$$

For Perception MCQs, EyePCR-MLLM reports the following accuracies across the seven perceptual views: V1 0.8991, V2 0.8768, V3 0.6872, V4 0.6776, V5 0.8339, V6 0.5452, and V7 0.6683, with an overall average of 0.7412. The principal comparison points are GPT-4.1 at 0.7069 average, Gemini 2.5 Pro at 0.7138, GPT-o3 at 0.6916, Claude 3.7 at 0.6530, Gemini 2.0 Flash at 0.6212, and Gemini 2.5 Flash at 0.6647. Among open-source baselines, Qwen2.5-VL-7B scores 0.3866, Qwen2.5-VL-32B 0.4036, InternVL-14B 0.4398, LLaVA-Video-7B 0.2380, and VideoLLaMA3-7B 0.2158. Human references are Ophthalmology Trainee at 0.6567 and Ophthalmic Surgeon at 0.9090.

For OEQs, EyePCR-MLLM reports Perception BLEU-4 0.1099 and ROUGE-L 0.3795; Comprehension 0.0513 and 0.2513; D1 0.0510 and 0.2355; D2 0.0703 and 0.2792; D3 0.0645 and 0.2638; and D4 0.0597 and 0.2612. Its overall averages are BLEU-4 0.0678, ROUGE-L 0.2784, and rubric 3.0062.

The aggregate comparisons are summarized below.

Evaluation regime	EyePCR-MLLM	Comparative context
MCQ Perception average	0.7412	GPT-4.1 0.7069; Gemini 2.5 Pro 0.7138; Qwen2.5-VL-7B 0.3866; Surgeon 0.9090
OEQ average	BLEU-4 0.0678; ROUGE-L 0.2784; rubric 3.0062	Gemini 2.5 Pro 0.0731 / 0.3293; GPT-o3 rubric 3.9349; Surgeon rubric 4.1583

These results establish two distinct empirical profiles. First, EyePCR-MLLM is the best open-source model on the reported Perception MCQs and surpasses several commercial models despite its 7B scale. Second, on generative OEQs it remains below the strongest commercial systems. Gemini 2.5 Pro attains the highest average BLEU-4 and ROUGE-L among the commercial models listed, with 0.0731 and 0.3293 respectively, while GPT-o3 attains the highest rubric score at 3.9349. Human surgeons achieve the highest rubric score overall at 4.1583, and the trainee score is 3.1697.

6. Strengths, limitations, and clinical significance

The principal strength reported for EyePCR-MLLM is fine-grained perceptual performance (Wang et al., 19 Sep 2025). It sets the best open-source MCQ accuracy with average 0.7412 and exceeds several commercial models. The paper highlights especially strong performance in anatomy and intraoperative structure views, V1 and V5. A second major strength is domain alignment: the average MCQ score improves from 0.3866 for the base Qwen2.5-VL-7B to 0.7412 for EyePCR-MLLM, indicating a large gain from domain instruction tuning on EyePCR. A third strength lies in structured grounding: the use of scene graphs and reasoning paths for dataset construction improves clinical validity and reduces hallucination in question generation.

The principal limitations arise in generative reasoning and in clinically subtle operating-room contexts. EyePCR-MLLM lags behind top commercial MLLMs on BLEU, ROUGE, and rubric scores for OEQs, and it struggles with long, context-rich clinical rationales and multi-hop planning or simulation. The paper also notes a gap relative to surgeons, which indicates difficulty with microanatomical variations and fine manipulations such as iris stability and capsule tears, where nuanced temporal context integration is required. This suggests that domain adaptation on a benchmark with video, subtitles, and structured knowledge is not by itself sufficient to close the expert-level gap in surgical cognition.

A further limitation is architectural. No explicit KG-fusion modules or domain adapters are introduced, which limits deeper use of surgical ontologies at inference time. Likewise, temporal modeling details are absent, even though the benchmark is video-based. The reported model therefore appears stronger at structured perceptual recognition than at explicitly grounded, long-horizon, clinically contingent reasoning.

In clinical-reliability terms, the PCR hierarchy is designed to simulate surgeon cognition: Perception captures multi-view fine-grained cues, Comprehension aligns scene graph and semantic memory, and Reasoning targets intent, flow, decisions, and risks. The remaining gap to surgeons implies that real-world deployment would require stronger temporal modeling, explicit knowledge integration, uncertainty handling, and safety checks. A plausible implication is that future systems in this line will need tighter model–KG coupling, long-horizon state tracking, and more explicit safety-oriented inference mechanisms before high-stakes intraoperative use becomes credible.

7. Research role and prospective development

EyePCR-MLLM occupies a specific position in multimodal surgical AI: it is both a benchmark-tuned baseline and a demonstration of the PCR paradigm for ophthalmic surgery (Wang et al., 19 Sep 2025). Its practical relevance is framed in three ways. As a training substrate, EyePCR’s multi-view attributes and KG-grounded VQAs provide a curriculum that moves models from seeing to understanding to deciding. As a clinical decision-support precursor, improved tool and anatomy perception together with intent inference could augment surgical education, intraoperative assistance, and risk awareness. As an evaluation instrument, EyePCR surfaces both perceptual and reasoning gaps under clinically grounded task definitions.

The paper’s reporting also leaves several open research questions. Because code, checkpoints, and ablations are absent, the relative contributions of data scale, instruction tuning, question construction, and any unreported implementation choices cannot be isolated. Because no statistical significance or confidence intervals are reported, the robustness of inter-model differences must be interpreted descriptively rather than inferentially.

Several future directions follow directly or plausibly from the reported limitations. A plausible implication is that larger and richer knowledge graphs, tighter model–KG integration modules for explicit multi-hop reasoning, and better temporal modeling through segment-to-procedure memory, causal inference, and long-horizon state tracking would directly target the observed OEQ weaknesses. Additional plausible extensions include uncertainty-aware reasoning, safety guardrails for high-stakes decisions, multimodal extensions using sensor data and instrument telemetry, and human-in-the-loop evaluation with continual learning from expert feedback. Within the scope of the reported study, EyePCR-MLLM therefore functions less as a final clinical system than as an empirical reference point for benchmarking and improving multimodal surgical cognition in ophthalmology.