FundusGen: Dataset for Ophthalmic Multimodal AI

Updated 4 July 2026

FundusGen is a dataset and instruction corpus that integrates global disease classification with region-level annotations and cognitive chain diagnostic reasoning.
It is generated via the Fundus-Engine pipeline, which automates fine-grained labeling, bounding box conversion, and GPT-4o–driven semantic expansion, ensuring clinically aligned outputs.
Empirical evidence demonstrates that FundusGen significantly boosts diagnostic accuracy and region grounding in multimodal MLLMs, outperforming conventional annotation strategies.

FundusGen denotes, in its formal and explicit usage, the ophthalmology-specific multimodal instruction-tuning dataset constructed through the Fundus-Engine system and used to fine-tune the multimodal LLM FundusExpert. It integrates global disease classification, local object detection, fine-grained feature analysis, and clinically aligned cognitive chain reasoning on approximately 200K color fundus images, with the stated goal of enabling “positioning-diagnosis collaboration” in ophthalmic multimodal LLMs (Liu et al., 23 Jul 2025). In a broader interpretive sense, the name has also been used “in spirit” to describe generic or generative fundus-image systems for enhancement, synthesis, anomaly generation, modality translation, and large-scale synthetic data production (Li et al., 2023, Zhao et al., 2017, Niu et al., 2023, Shang et al., 2023).

1. Formal definition and system context

In the terminology of FundusExpert, FundusGen is the dataset and instruction corpus, not the data-construction pipeline itself. The corresponding pipeline is called Fundus-Engine, and the model trained on the resulting corpus is FundusExpert, an ophthalmology-specific MLLM based on InternVL2.5-8B (Liu et al., 23 Jul 2025).

Component	Role	Main content
Fundus-Engine	Data construction system	Aggregation, localization, semantic expansion
FundusGen	Dataset and instruction corpus	~200K images, multi-granular labels, cognitive chains
FundusExpert	Fine-tuned MLLM	Positioning-diagnosis collaboration

This distinction is central. Fundus-Engine automates localization and semantic expansion; FundusGen stores the resulting multimodal supervision; FundusExpert consumes that supervision during instruction tuning. The dataset is designed to address two problems identified by the authors: fragmentation of annotation granularity and inconsistencies in clinical reasoning logic. FundusGen therefore couples region-level grounding with diagnostic reasoning sequences, rather than treating image classification, lesion localization, and report generation as unrelated tasks (Liu et al., 23 Jul 2025).

A plausible implication is that FundusGen should be understood less as a conventional benchmark dataset than as a structured supervision substrate for multimodal reasoning. Its unit of value is not only the image-label pair, but the joint packaging of image, region annotation, and clinically aligned textual instruction.

2. Data sources and annotation strata

FundusGen is built from approximately 200K color fundus images drawn from both public and in-house sources. The public component includes MM-Retinal, BRSET, IDRiD, APTOS2019, Messidor2, PAPILA, Retina (CataractDataset), and Glaucoma_fundus. The in-house component consists of high-quality color fundus images annotated by expert ophthalmologists with comprehensive global diagnoses and characteristic lesion labels (Liu et al., 23 Jul 2025).

The annotation schema is explicitly multi-granular. It combines global diseases, local structures and lesions, fine-grained visual traits, and textual descriptions intended for instruction tuning.

Annotation layer	Content	Examples
Global disease labels	Multi-label diagnoses and grading	DR, DME, glaucoma, cataract, AMD
Local lesion/structure labels	Region-level findings	Optic disc, optic cup, hard exudates, microaneurysms, cotton-wool spots
Fine-grained features	Morphology and spatial distribution	Size, shape, color, clustered in macular region
Textual annotations	Region-grounded clinical descriptions	Objective findings and implicit diagnostic clues

The global label space includes diabetic retinopathy with severity grading, diabetic macular edema, glaucoma, cataract, hypertensive retinopathy, age-related macular degeneration, and additional macular and rare diseases. The local annotation layer focuses on optic disc, optic cup, hard exudates, microaneurysms, and cotton-wool spots. Fine-grained descriptors include lesion morphology, spatial distribution, and structural characteristics such as optic disc cupping ratio and neuroretinal rim changes (Liu et al., 23 Jul 2025).

The textual layer is not free-form captioning in a generic sense. It is constrained to produce descriptions that are directly traceable to pixel-level or region-level evidence while also encoding clinically relevant implications. This suggests that FundusGen occupies an intermediate position between a detection dataset, a report corpus, and an instruction-tuning set.

3. Fundus-Engine pipeline and construction procedure

Fundus-Engine builds FundusGen through three stages: fine-grained label collection, automated bounding box annotation, and MLLM-driven semantic expansion (Liu et al., 23 Jul 2025).

The first stage aggregates approximately 200K fundus images with structured labels from open-source datasets and in-house collections. These labels include global disease information and, for selected datasets, pixel-level lesion or structure annotations. The second stage converts segmentation information into scalable localization supervision. It trains separate nnU-Net models for optic cup, optic disc, hard exudates, microaneurysms, and cotton-wool spots, using high-quality segmentation data after AutoMorph-style quality control. It then performs semi-supervised expansion by generating pseudo segmentation labels on additional images and iterating the process (Liu et al., 23 Jul 2025).

The paper reports cross-domain evidence for this pseudo-label strategy. For example, optic disc Dice improves from 75.3% using true labels to 82.0% using pseudo labels in the first round, while microaneurysm Dice improves from 21.1% to 25.0%. After segmentation, Fundus-Engine converts masks into boxes using DBSCAN with epsilon = 160, min_samples = 10, an area > 100 pixels filter, and retention of the top three boxes per lesion or structure (Liu et al., 23 Jul 2025).

This stage markedly expands localization density. The reported counts increase from 882 to 5,357 for microaneurysms, from 642 to 10,089 for hard exudates, from 291 to 1,876 for cotton-wool spots, from 901 to 16,551 for optic cup, and from 1,070 to 16,720 for optic disc (Liu et al., 23 Jul 2025). The significance is not only scale, but the conversion of sparse segmentation resources into region-grounded supervision suitable for MLLM instruction tuning.

The third stage performs semantic expansion. Structured outputs from the first two stages are normalized, then passed through prompt templates to GPT-4o, which generates region-grounded text describing abnormalities, structures, and diagnostic implications. The prompting is constrained by two principles: observational objectivity and clinical relevance. Ophthalmologists then perform double-blind review, discarding or regenerating inconsistent outputs (Liu et al., 23 Jul 2025). This human-in-the-loop filtering is an essential part of FundusGen’s claimed clinical alignment.

4. Cognitive-chain design and instruction-tuning tasks

A defining feature of FundusGen is the use of a clinically aligned cognitive chain, formalized as:

$\text{Region localization} \rightarrow \text{Feature analysis} \rightarrow \text{Diagnostic reasoning}$

This chain is encoded as multi-turn instruction data rather than post hoc explanation. The first class of interaction proceeds from local findings to global diagnosis: one turn asks for abnormal region analysis, and a later turn asks for a diagnostic suggestion based on those findings. The second class performs “verification and deepening of the evidence chain”: one turn gives a preliminary diagnostic analysis, and a later turn requests disease-specific evidential analysis, such as glaucoma-related optic disc and nerve fiber layer features (Liu et al., 23 Jul 2025).

FundusGen operationalizes this through five instruction families.

Task type	Function	Typical output
General Report	Standardized ophthalmic report generation	Global diagnostic report
Regional QA	Localization and region labeling	Bounding boxes or grounded region text
Grounding Report	Report with explicit positional grounding	Region-linked narrative
Multi-turn Diagnostic Reasoning	Local-to-global reasoning	Findings followed by diagnosis
Multi-turn Confirmation Analysis	Hypothesis verification	Preliminary diagnosis followed by disease-specific evidence

These task types jointly supervise localization, reporting, and reasoning. Regional QA directly maps coordinates to answers; Grounding Report forces text to reference image regions; the two multi-turn tasks encode temporal dependency between observations and conclusions (Liu et al., 23 Jul 2025).

The methodological importance of this design is that FundusGen does not treat reasoning as a purely textual add-on. It couples region grounding to dialogue structure. This suggests that, within the FundusExpert framework, interpretability is not defined only by verbal justification, but by a staged relationship between spatial evidence and diagnostic synthesis.

5. Empirical impact, scaling behavior, and ablation evidence

FundusGen functions as the primary supervision source for FundusExpert and also transfers effectively to other generalist MLLMs under unified instruction fine-tuning (Liu et al., 23 Jul 2025). FundusExpert achieves 69.7% accuracy on Fundus-MMBench and 66.7% on the GMAI-MMBench fundus subset. Relative to MedRegA (40B), the gains are +29.4% on Fundus-MMBench and +26.6% on GMAI-MMBench. Relative to GPT-4o, FundusExpert reaches 69.7 vs 41.6 on Fundus-MMBench and 66.7 vs 57.4 on GMAI-MMBench (Liu et al., 23 Jul 2025).

The dataset also supports strong region grounding. In zero-shot object detection evaluation, FundusExpert reaches IoU values of 0.632 for optic cup, 0.738 for optic disc, 0.194 for hard exudates, 0.141 for cotton-wool spots, and 0.116 for microaneurysms, substantially exceeding the compared MLLMs (Liu et al., 23 Jul 2025). In zero-shot report generation, FundusExpert reaches 77.0% clinical consistency versus 47.6% for GPT-4o. The paper defines clinical consistency through semantic matching between ground-truth labels and the generated report, normalized by the union of label and semantic feature sets (Liu et al., 23 Jul 2025).

FundusGen also exhibits a data-quality scaling law. When FundusExpert is fine-tuned with varying fractions of FundusGen, performance on GMAI-MMBench follows:

$L \propto N^{0.068}$

with $R^2 = 0.972$ , adjusted $R^2 = 0.930$ , and MSE = 0.0001 (Liu et al., 23 Jul 2025). By contrast, an alternative “Classification Annotation-Guided Data” corpus, generated without localization and without explicit cognitive chains, shows no clear scaling law. The authors further report that 10% of FundusGen yields performance comparable to 100% of that baseline corpus, which they interpret as evidence of higher data efficiency (Liu et al., 23 Jul 2025).

Ablation studies indicate that FundusGen’s structure, not only its size, matters. Breaking multi-turn cognitive chains into independent single-turn tasks reduces performance from 69.7% to 67.1% on Fundus-MMBench and from 66.7% to 63.2% on GMAI-MMBench. Removing region-aware instructions lowers performance from 68.9% to 65.3% on Fundus-MMBench and from 64.7% to 59.3% on GMAI-MMBench. Removing startup General Report data lowers performance from 66.2% to 62.9% on Fundus-MMBench and from 60.9% to 56.4% on GMAI-MMBench in the reported one-epoch setting (Liu et al., 23 Jul 2025).

The trained FundusExpert can also replace GPT-4o inside Fundus-Engine’s semantic expansion stage. When used to generate 100K new reports for unseen images, the resulting synthetic corpora lead to better downstream fine-tuning of Qwen2-VL-7B and InternVL2.5-8B than GPT-4o-generated data (Liu et al., 23 Jul 2025). This suggests a recursive data-engineering loop in which FundusGen-trained models become domain-specific data generators.

6. Broader interpretive usage in fundus image generation research

Outside the strict dataset sense, “FundusGen” has also been used interpretively to denote a generic or generative fundus-image framework. This broader usage is explicit in several technical narratives, but it is not a standardized formal name across those papers.

One line of work treats FundusGen as a generic enhancement or preprocessing block. GFE-Net is described as pursuing “what you might call ‘FundusGen’ in spirit”: a generic, plug-and-play enhancement module trained once on clear images with synthetic degradations, without real paired low-high images, unpaired real high-quality sets, segmentations, or test-domain data during training (Li et al., 2023). Another line treats it as conditional synthesis from vessel topology. “Synthesizing Filamentary Structured Images with GANs” is described as giving “almost exactly what you would want under the name ‘FundusGen’,” with Fila-GAN and Fila-sGAN generating realistic fundus images from vessel maps and demonstrating usefulness for downstream vessel segmentation, even with as few as 10 training examples (Zhao et al., 2017).

A second cluster of papers extends the idea toward synthetic anomaly generation and trait-conditioned generation. ReSynthDetect is presented as “exactly the kind of design relevant for a ‘FundusGen’-style framework,” combining a normal reconstruction model with a fundus-consistent anomaly generator for unsupervised anomaly detection (Niu et al., 2023). “Futuristic Variations and Analysis in Fundus Images Corresponding to Biological Traits” is described as proposing “a prototype of what you could call ‘FundusGen’,” with FGC-Net generating multiple age-conditioned variants of an input fundus image (Hassan et al., 2023).

A third cluster frames FundusGen as cross-modality generation. This includes color fundus to macular heightmap translation via conditional GANs (Tahghighi et al., 2020), multiple families of fundus-to-fluorescein-angiography synthesis systems (Kamran et al., 2020, Li et al., 2020, Kamran et al., 2020, Kamran et al., 2021), and ultra-wide-angle SLO-to-FA synthesis through multi-scale GANs with attention and registration enhancement (Fang et al., 2023, Ge et al., 2024). In these papers, the common theme is that non-invasive or lower-cost retinal modalities are used to synthesize clinically informative but invasive or expensive targets.

A fourth cluster uses the concept at dataset scale. ReTree employs a two-stage DDPM to generate vessel trees and then conditioned fundus images, producing 30,000 synthetic fundus-vessel pairs for vessel segmentation (Alimanov et al., 2023). SynFundus-1M releases over one million fundus images with fifteen types of annotation, generated by a conditional diffusion model trained on more than 1.3 million private authentic fundus images (Shang et al., 2023). In this broader sense, FundusGen denotes a family of controllable fundus generation systems rather than a single dataset.

7. Limitations and future directions

FundusGen, in its formal dataset sense, has several limitations acknowledged at the system level. Disease coverage remains finite and is restricted to fundus photography; other ophthalmic modalities such as OCT and fundus autofluorescence are not included. The data sources span different countries and devices, but this also introduces potential demographic and device bias. Region annotations depend partly on pseudo-labels produced by nnU-Net, and the reported segmentation quality is imperfect for small lesions such as microaneurysms and cotton-wool spots. The textual layer initially depends on GPT-4o, which can introduce subtle semantic bias or hallucination despite expert review. Finally, the clinical cognitive chain itself is hand-crafted and may reflect a particular reasoning style rather than the full diversity of ophthalmic diagnostic practice (Liu et al., 23 Jul 2025).

The future directions proposed around FundusGen are correspondingly systemic. The authors point to reinforcement learning on FundusGen’s semantic hierarchy, dynamic reasoning and test-time scaling, extension of Fundus-Engine to other ophthalmic modalities and clinical scenarios, and expansion of pseudo-labeling to more lesions and structures (Liu et al., 23 Jul 2025). A plausible implication is that FundusGen may evolve from a fundus-only instruction corpus into a multimodal ophthalmic data engine with region grounding, structured reasoning, and domain-specific synthetic annotation loops.

In the broader interpretive literature, related future directions include stronger structure-preserving enhancement backbones, more advanced frequency modeling, learned lesion generators, diffusion-based refinement, style disentanglement, and broader cross-modality generation (Li et al., 2023, Niu et al., 2023). Taken together, these trajectories suggest that “FundusGen” now names both a specific dataset in ophthalmic MLLM research and a wider design ambition: a clinically aligned, controllable, and scalable generative infrastructure for fundus-centered ophthalmic AI.