EyeQ: Multidomain Research Benchmarks

• EyeQ is a multifaceted research name linked to distinct artifacts in retinal imaging, vision-language reasoning, and software security.
• In retinal imaging, EyeQ provides a detailed fundus-image dataset with explicit quality grading, supporting diagnostic quality assessment, image enhancement, and anomaly detection.
• Eye-Q in vision-language and EyeQ in software security offer multilingual puzzle solving benchmarks and automated fuzzing systems, underscoring the need for domain-specific disambiguation.

EyeQ is an overloaded research name applied to several distinct artifacts. In ophthalmic imaging, it denotes a retinal fundus image dataset and benchmark derived from EyePACS, used for diagnostic quality assessment, diabetic retinopathy grading under image degradation, image enhancement, and unsupervised anomaly detection (Fu et al., 2019). In vision-language research, the closely styled “Eye-Q” denotes a multilingual benchmark for visual word puzzle solving and image-to-phrase reasoning (Najar et al., 6 Jan 2026). In software security, EyeQ denotes a code review-guided fuzzing system that converts review comments into IJON-based annotations for AFL++ (Luu et al., 11 Feb 2026). The shared name therefore requires domain-specific disambiguation.

Name	Domain	Object
EyeQ	Retinal imaging	Fundus-image dataset and benchmark
Eye-Q	Vision-language reasoning	Multilingual puzzle benchmark
EyeQ	Software security	Code review-guided fuzzing system

1. EyeQ in retinal imaging: dataset definition and annotation

The retinal EyeQ dataset was introduced as a re-annotation of EyePACS retinal images for retinal image quality assessment (RIQA). It contains 28,792 color fundus photographs after discarding ambiguous cases and uses a three-level quality grading system: Good, Usable, and Reject (Fu et al., 2019). The quality criteria are clinically explicit. Good images have no visible low-quality artifacts and sharply show the optic disc, macula, and major vessels; Usable images may contain blur or illumination/contrast artifacts affecting up to 20% of the fundus area while preserving interpretability of key structures; Reject or Unusable images have severe blur, very low contrast, heavy artifacts, or missing key structures such as the optic disc or macula, making reliable diagnosis impossible (Fu et al., 2019, Manne et al., 2023).

The annotation protocol is reported somewhat differently across subsequent EyeQ-based studies. One report states that two expert graders independently labeled all EyePACS images into Good, Usable, or Reject, and that discordant labels were excluded; no formal inter-rater $\kappa$ was reported (Fu et al., 2019). Another states that images were graded by experts into Good, Usable, and Unusable according to predefined rules, while inter-observer agreement statistics were not reported (Manne et al., 2023). A later semi-supervised study preserved the overall-quality labels and added a newly released subset, EyeQ-D, of 160 test images manually annotated for illumination, clarity, and contrast by 8 ophthalmologists, with majority vote as the final label (Telesco et al., 17 Nov 2025).

The split statistics are likewise not uniform across the literature. One report gives 12,543 training images and 16,249 testing images, with training counts of 8,347 Good, 1,876 Usable, and 2,320 Reject, and test counts of 8,470 Good, 4,559 Usable, and 3,220 Reject (Fu et al., 2019). Another gives the same total split sizes but reports 6,308 Good, 4,355 Usable, and 1,880 Unusable in training, and 7,826 Good, 4,491 Usable, and 3,932 Unusable in testing (Manne et al., 2023). This suggests that EyeQ citations should specify the exact experimental protocol rather than assuming a single canonical class distribution.

2. Diagnostic quality assessment on EyeQ

EyeQ’s initial role was RIQA. Fu et al. proposed the Multiple Color-space Fusion Network (MCF-Net), motivated by the claim that RGB alone underuses information available in alternative color-spaces. MCF-Net processes RGB, HSV, and LAB representations with separate CNN backbones, concatenates the resulting feature vectors, produces an intermediate feature-fusion prediction, and then fuses branch-level and feature-level predictions into a final 3-way classifier. With a DenseNet121 backbone, the full model reported Accuracy $= 0.9175$, Precision $= 0.8645$, Recall $= 0.8497$, and $F_1 = 0.8551$, outperforming the hand-crafted-feature baseline and branch-only ablations (Fu et al., 2019).

A later hierarchical deep-learning formulation treated diagnostic quality as a continuum and decomposed the task into binary stages: High vs. Low DQ, then Good vs. Usable within High, and Usable vs. Unusable within Low. The best configuration used DenseNet-121 for Model-1 and Model-2 and EfficientNet-B0 for Model-3, trained with binary cross-entropy and an $L_2$ regularization term. On EyeQ, this system achieved Accuracy $= 89.44\%$, Precision $= 87.98\%$, Recall $= 87.70\%$, and $F$-Score $= 0.9175$0, with per-class recalls of $= 0.9175$1 for Good, $= 0.9175$2 for Usable, and $= 0.9175$3 for Unusable (Manne et al., 2023). The same work used Grad-CAM, reporting that Good images emphasized the optic disc, macula, and main vessels, while Unusable images emphasized blur and uneven illumination regions.

Interpretability was extended further by a semi-supervised multi-task framework that augmented overall quality prediction with pseudo-labels for illumination, clarity, and contrast. The student model used a shared ResNet-18 backbone with a 3-class softmax head for overall quality and a three-sigmoid head for detail prediction. Pseudo-labels were generated by a teacher trained on MSHF, then used for multi-task fine-tuning on EyeQ. On the EyeQ test set, the multi-task model improved over the single-task baseline from macro-$= 0.9175$4 to $= 0.9175$5, and from Accuracy $= 0.9175$6 to $= 0.9175$7; the largest per-class gain was in the ambiguous Usable category, whose $= 0.9175$8 rose from $= 0.9175$9 to $= 0.8645$0 (Telesco et al., 17 Nov 2025). On EyeQ-D, the quality-detail predictions were statistically indistinguishable from the teacher for most detail tasks ($= 0.8645$1), which the paper interprets as pseudo-label noise aligning with expert variability.

3. Enhancement and robust diagnosis under low image quality

EyeQ has also been used to study image restoration. An optimal transport-guided unsupervised enhancement framework treated low-quality-to-high-quality fundus enhancement as unpaired image-to-image translation from the Reject domain to the Good domain. The generator was a U-Net–inspired encoder-decoder with skip connections and Efficient Channel Attention modules; the discriminator was a PatchGAN-style critic with gradient penalty. The sample-wise consistency term was defined by multi-scale SSIM rather than pixelwise $= 0.8645$2 or $= 0.8645$3, with the aim of preserving vessels, optical discs, and lesions (Zhu et al., 2023).

On EyeQ, this framework reported a Converted Ratio of $= 0.8645$4, compared with $= 0.8645$5 for CycleGAN and $= 0.8645$6 for OTT-GAN, alongside downstream DR-classification Accuracy $= 0.8645$7, Cohen’s $= 0.8645$8, and AU-ROC $= 0.8645$9 on enhanced images (Zhu et al., 2023). On a synthetic full-reference test set, it reported PSNR $= 0.8497$0 and SSIM $= 0.8497$1, exceeding the baselines. Qualitatively, the method was described as maintaining vessel calibers and lesion contrast better than CycleGAN’s smoothing and OTT-GAN’s occasional vessel hallucinations.

A separate line of work used EyeQ to evaluate disease diagnosis under systematically varied low-quality proportions. The clinical-oriented multi-level contrastive learning framework (CoMCL) collapsed EyeQ’s original three quality labels into High-quality and Low-quality groups, used lesion and healthy patches extracted at $= 0.8497$2, and constructed contrastive pairs across lesion/healthy and high-/low-quality strata. Under the original EyeQ condition of $= 0.8497$3 low-quality images, CoMCL reported $= 0.8497$4 and Accuracy $= 0.8497$5 for DR grading, exceeding ResNet-50, DenseNet-121, MMCNN, Zoom-in-Net, Lesion-aware CL, and other baselines (Hou et al., 2024). Under synthetic degradations that raised the low-quality proportion to $= 0.8497$6 and $= 0.8497$7, CoMCL still reported the highest results, with $= 0.8497$8 of $= 0.8497$9 and $F_1 = 0.8551$0, respectively. The paper’s t-SNE and Grad-CAM analyses were used to argue that CoMCL better separates lesion semantics from low-quality confounds.

4. EyeQ in unsupervised fundus anomaly detection

EyeQ has additionally been used as an anomaly-detection benchmark. In “ReSynthDetect: A Fundus Anomaly Detection Network with Reconstruction and Synthetic Features,” Niu et al. used a good-quality subset of EyeQ, described as a filtered subset of EyePACS. Their setup used 6,324 DR grade 0 images as training normals and a test set of 8,470 images spanning grades 0–4, with each image resized to $F_1 = 0.8551$1 and preprocessed using CLAHE with ClipLimit $F_1 = 0.8551$2 and GridSize $F_1 = 0.8551$3 (Niu et al., 2023).

ReSynthDetect is a two-stage model. Stage I trains an autoencoder on normal images only using the reconstruction loss

$F_1 = 0.8551$4

After convergence, the encoder is frozen and its decoder discarded. Stage II uses a U-shaped localization network with skip connections; at each layer, reconstructive features from the frozen encoder and localization features from the trainable encoder are concatenated and fed forward. Synthetic anomaly supervision is provided through Perlin-based masks and self-mix blending, and training uses a focal loss with focusing parameter $F_1 = 0.8551$5 rather than an adversarial term (Niu et al., 2023).

For image-level scoring, the method averages the top-10 highest pixel anomaly scores per image. On EyeQ, the reported image-level AUROC for 0 vs all was $F_1 = 0.8551$6, compared with $F_1 = 0.8551$7 for fAnoGAN, $F_1 = 0.8551$8 for MKD, $F_1 = 0.8551$9 for DRAEM, and $L_2$0 for Lesion2Void (Niu et al., 2023). Grade-specific breakdowns were $L_2$1 for 0 vs 1, $L_2$2 for 0 vs 2, $L_2$3 for 0 vs 3, and $L_2$4 for 0 vs 4. The paper attributes the gain to consistent synthetic anomalies, reconstruction-based feature guidance, the two-stage normality-then-deviation design, and focal loss; this suggests that EyeQ can serve not only as a quality benchmark but also as a substrate for modeling subtle pathological deviation from normal retinal anatomy.

5. “Eye-Q” as a multilingual image-to-phrase reasoning benchmark

In a separate literature, “Eye-Q” designates a multilingual benchmark for visual word puzzle solving rather than a retinal dataset. It contains 1,343 puzzles across English (300), Persian (671), Arabic (50), and cross-lingual (322) subsets, each consisting of an image, a short prompt describing the puzzle objective, and a unique ground-truth target word or short phrase in the required script (Najar et al., 6 Jan 2026). The benchmark is designed to probe implicit cue discovery, relational abstraction, and linguistic association, with puzzle types including charades, rebus-style symbol substitution, phonetic or orthographic substitution, and culturally grounded idioms.

The task is open-ended generation: the model must infer the exact target word or phrase without answer options. Evaluation uses exact-match accuracy,

$L_2$5

with additional cosine-similarity analysis in a text-embedding space to test whether failures are merely surface mismatches (Najar et al., 6 Jan 2026). The prompting protocol includes a basic prompt with an answer-length hint, three few-shot chain-of-thought examples, up to two iterative refinements after an incorrect answer, and a partial character reveal exposing $L_2$6 of the answer’s non-space characters.

Benchmarking of GPT-5.2, Gemini 2.5 Flash, Gemini 2.5 Pro, Grok 4.1 Fast reasoning, Llama 4 Scout, and Qwen 3 VL showed low aggregate performance: the Basic setup averaged $L_2$7, iterative refinement $L_2$8, and partial reveal $L_2$9 (Najar et al., 6 Jan 2026). By subset, the reported best results were $= 89.44\%$0 on English, $= 89.44\%$1 on Persian, $= 89.44\%$2 on Arabic, and $= 89.44\%$3 on cross-lingual puzzles. The paper reports maximum accuracy of only $= 89.44\%$4 overall and observes that many failures are semantically distant from the target, indicating mis-selection of cues or incorrect abstraction rather than minor wording differences.

6. EyeQ as a code review-guided fuzzing system

In software security, EyeQ denotes a workflow for converting code review discussions into fuzzing guidance. The system comprises four stages: Review Classification, Code Localization, Code Instrumentation, and Annotation-aware Fuzzing (Luu et al., 11 Feb 2026). Security-relevant review comments are identified and optionally mapped to CWE categories; localized comments are then associated with modified functions; IJON annotations such as IJON_SET, IJON_MAX, IJON_MIN, IJON_STATE, IJON_DIST, and IJON_CTX are inserted; finally, the instrumented code is fuzzed with AFL++ plus IJON.

The formalization treats each annotation as a reward-producing function over executions, augmenting ordinary coverage with semantic novelty. In practice, EyeQ stores annotation signals in a separate bitmap region so that AFL++ can schedule seeds that produce new edge coverage or new annotation behavior (Luu et al., 11 Feb 2026). The automated version uses Gemini-2.5-flash, few-shot prompting, hard prompt constraints, and a Python-based toolchain for GitHub retrieval, localization, instrumentation, and Dockerized AFL++ campaigns.

Evaluation was conducted on the PHP interpreter. On a 234-comment dataset drawn from 2011–2022 PHP pull requests, human-guided EyeQ mapped 37 of 117 security comments to 34 commits; over 24-hour fuzzing runs per commit, AFL++ alone found 7 unique crashes, whereas human-guided EyeQ found 31 (Luu et al., 11 Feb 2026). After porting to the latest PHP version and fuzzing for four months, 41 previously unknown bugs were confirmed by maintainers, including 27 fixed upstream. On the same 234-comment dataset, the automated system achieved $= 89.44\%$5 agreement with human labels for review classification, $= 89.44\%$6 recall of human-localized functions, and $= 89.44\%$7 intent match for instrumentation, yet found 47 unique crashes in end-to-end fuzzing compared with 31 for EyeQ$= 89.44\%$8 and 7 for AFL++.

7. Disambiguation and cross-domain significance

The name EyeQ therefore refers not to a single benchmark or system but to at least three unrelated research objects. In medical imaging, EyeQ is a public benchmark built around retinal image quality labels and subsequently extended into enhancement, DR-robust diagnosis, anomaly detection, and interpretable quality-detail prediction (Fu et al., 2019). In vision-language research, Eye-Q is a benchmark for open-ended puzzle solving across English, Persian, Arabic, and cross-lingual settings (Najar et al., 6 Jan 2026). In software security, EyeQ is a method for translating code review intelligence into annotation-aware fuzzing guidance (Luu et al., 11 Feb 2026).

A common misconception is to treat these usages as variants of one benchmark family. The available literature does not support that interpretation. The retinal EyeQ uses fundus photographs and quality or pathology labels; Eye-Q uses puzzle images and exact phrase answers; the fuzzing EyeQ operates on code reviews, program diffs, IJON annotations, and crash outcomes. This suggests that citations to “EyeQ” should always include domain context, and often the arXiv identifier, to avoid conflating distinct lines of work that share only a name.