GenExam: Exam Benchmark for Text-to-Image

• GenExam is a multidisciplinary benchmark that uses exam-style prompts to test text-to-image systems on domain-specific reasoning and precise visual rendering.
• It comprises 1,000 real exam questions across ten subjects and a four-level taxonomy, using fine-grained scoring for semantic correctness and presentation quality.
• Results highlight that even advanced models struggle with strict criteria, emphasizing the need for improved integration of domain knowledge, symbolic reasoning, and exact rendering.

Searching arXiv for GenExam and closely related benchmark/model papers to ground the article in current literature. arXiv search query: "GenExam benchmark text-to-image exam multidisciplinary" GenExam is a multidisciplinary text-to-image exam benchmark that evaluates whether image generation systems can satisfy rigorous, discipline-specific drawing requirements rather than merely produce visually plausible illustrations. It introduces 1,000 real exam-style questions with ground-truth images across ten subjects, organized under a four-level taxonomy, and equips each problem with fine-grained scoring points for semantic correctness and visual plausibility. In this formulation, image generation is framed as an examination of integrated understanding, domain reasoning, and precise rendering, with the benchmark positioned as the first benchmark for multidisciplinary text-to-image exams (Wang et al., 17 Sep 2025).

1. Benchmark rationale and problem formulation

GenExam is motivated by the observation that human school and university exams test not only knowledge recall but also integrated understanding, domain reasoning, and precise drawing skills. The benchmark transfers this exam structure to text-to-image generation. Its prompts resemble real exam questions and impose strict, unambiguous constraints such as exact coordinates, labels, color-coding, and structural relations. This design departs from benchmarks centered on generic illustration or world-knowledge depiction and instead evaluates rigorous drawing exams in domains such as molecular diagrams, geometric constructions, circuit schematics, and data charts (Wang et al., 17 Sep 2025).

The benchmark is explicitly multidisciplinary. It covers mathematics, physics, chemistry, biology, computer science, engineering, geography, economics, history, and music. The taxonomy is organized under a four-level ISCED-F–based hierarchy, enabling evaluation at multiple granularities from broad subject to specific topic. GenExam therefore functions simultaneously as a generation benchmark and as a structured curriculum over academic visual tasks.

A common misunderstanding is to treat GenExam as a benchmark of generic prompt following. The benchmark definition is narrower and more exacting: prompts are exam-style, each sample has a reference image, and correctness is judged through fine-grained rubric items rather than by informal visual resemblance. Another possible misunderstanding is to view the taxonomy as a cognitive taxonomy in the Bloom sense. GenExam does not sort problems by cognitive level in the Bloom’s sense; instead it uses a four-level subject hierarchy (Wang et al., 17 Sep 2025).

2. Dataset composition and taxonomic structure

GenExam contains 1,000 real exam-style questions with ground-truth images. The ten top-level subjects and their counts are as follows (Wang et al., 17 Sep 2025).

Subject	Count
Mathematics	151
Physics	113
Chemistry	118
Biology	156
Computer Science	102
Engineering	111
Geography	66
Economics	77
Music	65
History	41

The four-level taxonomy provides a subject hierarchy rather than a flat label set. A representative example from chemistry is: Level 1, Chemistry; Level 2, Structure of Matter; Level 3, Molecular Structure; Level 4, Organic Compounds. Across the full benchmark, the hierarchy comprises 10 top-level subjects, 40 level-2 fields, 132 level-3 topics, and 236 level-4 itemizations (Wang et al., 17 Sep 2025).

The subject descriptions delineate the intended visual reasoning regime. Mathematics includes analytic and plane/solid geometry, function plots, conic sections, and calculus-related graphs. Physics includes circuits, mechanics, optics, electromagnetism, thermodynamics, and quantum sketches. Chemistry covers atomic and molecular structure, reaction mechanisms, equilibrium curves, and organic or inorganic structures. Biology includes cell anatomy, molecular pathways, organ cross-sections, ecological food webs, and genetics pedigrees. Computer science includes trees, graphs, ER diagrams, digital logic, CPU pipeline diagrams, and network topologies. Engineering includes mechanical part schematics, structural-drawing views, surveying maps, and thermodynamic cycles. Geography, economics, history, and music similarly emphasize domain-constrained maps, curves, charts, territorial diagrams, and notation (Wang et al., 17 Sep 2025).

The benchmark also reports a difficulty binning by subject knowledge: easy at 24%, medium at 38%, and hard at 38%. This suggests that GenExam was designed not merely to sample broad coverage, but to distribute prompts across a nontrivial range of disciplinary demands.

3. Ground truth, annotation protocol, and scoring metrics

Each GenExam problem is paired with a reference image and fine-grained scoring points. The reference images are sourced or redrawn from open exam repositories such as AP, IB, and A-level materials and are vetted by PhD annotators. In the detailed benchmark description, each prompt is paired with 3–14 scoring “questions,” each carrying a fractional score $s_i$ with $\sum_i s_i = 1.0$; the abstract description characterizes the benchmark as using 4–14 fine-grained scoring points. The operational role is the same in both descriptions: a deterministic decomposition of correctness into explicit requirements (Wang et al., 17 Sep 2025).

Semantic correctness is defined as

$$C=\sum_{i=1}^{N} s_i\,\mathbf{1}[\text{answer}_i=\text{Yes}] \quad \in [0,1].$$

A multimodal LLM, specified as GPT-5 in the benchmark description, is prompted to answer each scoring question as Yes or No by comparing the generated image to the ground truth (Wang et al., 17 Sep 2025).

Visual plausibility is evaluated along three auxiliary dimensions, each in $\{0,1,2\}$: Spelling $S$, Logical Consistency $L$, and Readability $R$. Final benchmark metrics are then defined by two complementary aggregates:

• Strict score: the fraction of generated images with $C=1$ and $S=L=R=2$.
• Relaxed score:

$$\text{Relaxed} = 0.7\,C +0.1\,\frac{S}{2}+0.1\,\frac{L}{2}+0.1\,\frac{R}{2}\quad\in[0,1].$$

This metric design separates semantic compliance from presentation quality while preserving an exacting all-requirements criterion through the strict score. The separation is important because GenExam is intended to measure both whether the correct scientific or disciplinary content is present and whether the resulting visual artifact is spelled correctly, logically coherent, and legible.

4. Baseline performance and observed failure modes

The original GenExam study reports that even state-of-the-art systems perform poorly under strict grading, with most models yielding almost 0% strict and leading closed models remaining below 15% strict overall (Wang et al., 17 Sep 2025).

Model	Strict (%)	Relaxed (%)
GPT-Image-1 (GPT-4o)	12.1	62.6
Seedream 4.0	7.2	53.0
Imagen-4-Ultra	6.9	53.4
Gemini-2.5-Flash-Image	3.9	55.9

Open-source text-to-image and multimodal systems are summarized as scoring approximately 0% strict and 7–27 relaxed. For GPT-Image-1, subject-wise strict performance is highest in Biology at 22.8% and Physics at 13.2%, and lowest in History at 2.4% and Music at 9.3% (Wang et al., 17 Sep 2025).

Performance also degrades with benchmark difficulty. The reported strict and relaxed scores by difficulty are 18.6 and 68.1 for easy prompts, 12.8 and 62.2 for medium prompts, and 9.9 and 58.5 for hard prompts. This indicates that models do not fail only on obscure subdomains; rather, the benchmark exposes a systematic weakness in integrating domain knowledge, symbolic reasoning, and exact rendering.

The failure modes reported in the benchmark are concrete and discipline-specific. Semantic misses include wrong atom counts, misplaced tangents, and incorrect network topology. Spelling glitches include errors such as “Trnasposition” instead of “Transposition” in music diagrams. Logical inconsistency includes cases such as a point labelled $\sum_i s_i = 1.0$0 plotted on the $\sum_i s_i = 1.0$1-axis. Readability issues include overlapping labels in complex circuit schematics. Because strict score requires full semantic satisfaction together with maximal auxiliary quality, many systems that can produce aesthetically plausible diagrams still fail the exam-style criterion.

5. The GenExam-math subset and geometry-specific evaluation

A geometry-focused derivative of the benchmark, GenExam-math, is used in later work on exact rendering through constraint-engine interaction. GenExam-math is the subset of GenExam covering Euclidean-geometry diagram generation, with $\sum_i s_i = 1.0$2 problems distributed across plane geometry ($\sum_i s_i = 1.0$3), analytic geometry ($\sum_i s_i = 1.0$4), and solid geometry ($\sum_i s_i = 1.0$5). The problems are grouped into 37 taxonomies, including examples such as Triangle $\sum_i s_i = 1.0$6 Right_Triangle, Circle $\sum_i s_i = 1.0$7 Tangent, Analytic $\sum_i s_i = 1.0$8 Quadratic_Function, and Solid $\sum_i s_i = 1.0$9 Section_Straight_Cut (Hu et al., 20 May 2026).

For this subset, rendering is evaluated through semantic correctness and the same three auxiliary dimensions of readability, logical consistency, and spelling. In the GenExam-math formulation, the strict rendering score is

$$C=\sum_{i=1}^{N} s_i\,\mathbf{1}[\text{answer}_i=\text{Yes}] \quad \in [0,1].$$0

and the relaxed rendering score is

$$C=\sum_{i=1}^{N} s_i\,\mathbf{1}[\text{answer}_i=\text{Yes}] \quad \in [0,1].$$1

This is algebraically aligned with the benchmark-wide relaxed metric.

On GenExam-math, Draw2Think reports a strict rendering score of 68.2% and a relaxed rendering score of 90.5%. Against leading closed-source single-pass text-to-image baselines, the reported strict scores are 56.3% for Nano Banana 2, 50.3% for GPT-Image-2, and 47.0% for Seedream 5.0, with corresponding relaxed scores of 87.8%, 85.2%, and 82.9%. The largest reported gain is +11.9 percentage points strict over Nano Banana 2 and +5.3 percentage points relaxed over GPT-Image-2 (Hu et al., 20 May 2026).

The analysis of taxonomy-level behavior is also diagnostic. Best-served taxonomies, with 100% strict, include Triangle $$C=\sum_{i=1}^{N} s_i\,\mathbf{1}[\text{answer}_i=\text{Yes}] \quad \in [0,1].$$2 Perpendicular_Bisector, Circle $$C=\sum_{i=1}^{N} s_i\,\mathbf{1}[\text{answer}_i=\text{Yes}] \quad \in [0,1].$$3 Inscribed/Circumscribed, and Analytic $$C=\sum_{i=1}^{N} s_i\,\mathbf{1}[\text{answer}_i=\text{Yes}] \quad \in [0,1].$$4 Trigonometric_Function. Struggling taxonomies, with 0% strict, include many solid-geometry types such as Section $$C=\sum_{i=1}^{N} s_i\,\mathbf{1}[\text{answer}_i=\text{Yes}] \quad \in [0,1].$$5 Straight_Cut and Prism $$C=\sum_{i=1}^{N} s_i\,\mathbf{1}[\text{answer}_i=\text{Yes}] \quad \in [0,1].$$6 Oblique_Prism. Reported failures are partitioned into semantic-only failures (17/48 strict-fail), quality-only failures (7/48), combined failures (14/48), and timeouts (5/48). Solid geometry remains particularly difficult: 9/11 solid problems scored strict $$C=\sum_{i=1}^{N} s_i\,\mathbf{1}[\text{answer}_i=\text{Yes}] \quad \in [0,1].$$7 and relaxed $$C=\sum_{i=1}^{N} s_i\,\mathbf{1}[\text{answer}_i=\text{Yes}] \quad \in [0,1].$$8 (Hu et al., 20 May 2026).

6. GenExam as an evaluation target for reasoning-conditioned scientific image generation

GenExam has rapidly become a reference benchmark for models that combine domain reasoning with image synthesis. In one such system, S1-Omni-Image, the benchmark is used to evaluate a unified model for scientific image understanding, generation, and editing under a think-before-generate paradigm. The reported GenExam scores are 23.8 for Qwen-Image, 46.1 for GLM-Image, and 53.2 for S1-Omni-Image, with closed-source comparison points of 93.8 for GPT-Image-2 and 92.6 for Nano Banana 2. The paper attributes the improvement to explicit reasoning traces and a reasoning-to-diffusion alignment layer that helps organize scientific modules, arrow flows, labels, and hierarchical relations (Li et al., 23 Jun 2026).

GenExam is also used to assess the effect of large-scale structured academic visual data. DisciplineGen-1M introduces a 1.2M-sample multidisciplinary dataset and reports GenExam relaxed scores by subject and in aggregate for an open-source reasoning-generation system. The reported overall relaxed score is 51.4, compared with 42.9 for FLUX.2-dev, 35.8 for Qwen-Image-2512, and 13.5 for BAGEL. The reported gains are especially large on Chemistry (+20.5 over Qwen-Image-2512), History (+23.9), Engineering/Economics (+11.6/+17.2), and Biology (+7.1). By image type, the largest improvements over Qwen-Image are reported for Plots & Charts (+21.7), Chemical Structures (+16.7), Diagrams (+16.3), and Other (+46.7) (Wang et al., 2 Jul 2026).

These later results collectively suggest a specific research trajectory around GenExam. Models that first externalize or compute an explicit plan, and models trained on structured academic visual corpora, tend to improve more on the benchmark than generic image generators. A plausible implication is that GenExam rewards systems that bind disciplinary semantics, symbolic structure, and spatial layout before or during rendering, rather than treating diagram generation as unconstrained visual synthesis.

7. Significance, interpretation, and future directions

GenExam reveals a large gap between general text-to-image performance and precise domain-specific rendering. The benchmark’s central finding is not merely that current systems make occasional mistakes, but that even leading models struggle to satisfy all rubric items in an exam-style setting. This is particularly visible in strict score, where any unmet semantic requirement or any deficit in spelling, logical consistency, or readability prevents success (Wang et al., 17 Sep 2025).

The benchmark also provides a structured way to reason about what “knowledge-grounded” image generation requires. The reported results indicate persistent weaknesses in structured domain knowledge, symbolic reasoning, and rendering constraints such as label placement and color-coding. Later work proposes several directions that are consistent with this diagnosis: tighter fusion of reasoning chains into the generation pipeline, larger structured academic datasets, more fine-grained automated rubric scoring, specialized symbolic verifiers such as chemistry toolkits and geometry checkers, better text-rendering fidelity, reinforcement learning over generation preferences, and expansion to additional scientific modalities (Wang et al., 17 Sep 2025).

In this sense, GenExam functions as more than a benchmark leaderboard. It is a specification of a failure regime for current image generation systems: the inability to reliably compose expert knowledge, formal structure, and visual execution under explicit grading criteria. That specification has already made it useful both for evaluating closed and open models and for motivating architectures that place reasoning, verification, or structured supervision before image synthesis.