InterGenEval: Generative Evaluation Framework

• InterGenEval is a collection of evaluation methods that test if generative models preserve intended structural and semantic relationships through decomposed criteria.
• It is applied across adaptive assessment, interaction-aware video generation, and genetic programming, demonstrating versatility in structured system evaluation.
• By replacing holistic scoring with explicit, skill-level and aspect-decomposed measurements, InterGenEval enhances reliability and interpretability of model outputs.

InterGenEval is a research term used across several recent literatures for evaluation constructs that probe whether generative systems preserve intended structure under scoring, grounding, routing, or selection. In LLM-enabled adaptive assessment, it is used as an alias for Generative-Evaluative Agreement (GEA), the requirement that an LLM’s evaluative scoring function recover the skill levels that its generative function was instructed to produce (Lee et al., 19 May 2026). In interaction-aware video generation, it denotes a protocol for measuring whether a model correctly realizes subject–verb–object interactions and maintains those bindings over time (Jin et al., 8 Oct 2025). In adjacent work, the term is also used more loosely for interoperable evaluation layers for generative models, and, in Geometric Semantic Genetic Programming (GSGP), for a multi-generational selection mechanism that exploits archived populations at essentially zero additional memory cost (Castelli et al., 2022).

1. Terminological scope

The term has no single universal definition across arXiv usage. Instead, it appears in multiple domain-specific forms, all centered on structured evaluation of generative outputs.

Context	Meaning of InterGenEval	Core objects
LLM adaptive assessment	Generative-Evaluative Agreement	$\boldsymbol{\theta}, G, E, r, b, R^2$
Interaction-aware video generation	QA-based protocol for interaction fidelity	KISA, SGI, SPI, IF
Standardized benchmarking frameworks	Explicit spaces, aspects, or grounding rules	metrics, rubrics, conditions
GSGP	Multi-generational selection over archived populations	generations, tournaments, semantics

This suggests that InterGenEval functions less as a single benchmark name than as a family of evaluation ideas built around internal consistency, decomposition into interpretable units, and explicit measurement of failure modes.

2. InterGenEval as Generative-Evaluative Agreement in adaptive assessment

In the adaptive-assessment setting, the motivating problem is a self-referential validation loop: the same LLM dynamically generates assignment items, simulates student responses, and scores those responses. The formal validity requirement is that the evaluative function recover the skill levels that the generative function was instructed to realize. With a $k$-dimensional skill vector, here $k=24$, the paper defines a generative function $G:\boldsymbol{\theta}\mapsto x$ and an evaluative function $E:x\mapsto \hat{\boldsymbol{\theta}}$, and states the criterion as $E\circ G \approx I$ on the image of $\boldsymbol{\theta}$, equivalently

$$\mathbb{E}\big[\hat{\boldsymbol{\theta}} \mid x \sim G(\boldsymbol{\theta})\big] \approx \boldsymbol{\theta}.$$

Agreement is measured through per-dimension correlation $r_i=\mathrm{corr}(\theta_i,\hat{\theta}_i)$, pooled correlation $r$, mean bias $k$0, mean squared error, and a linear calibration model $k$1. The proposed benchmarks are strong InterGenEval for $k$2, moderate InterGenEval for $k$3, and a recommendation that skills with $k$4 not drive adaptive routing without human validation (Lee et al., 19 May 2026).

The direct measurement was performed in a Telegram-delivered adaptive coding assessment with two stages, each containing two assignments. The routing rule sent a candidate to a Stage 2 High path if the cumulative Stage 1 score met or exceeded a threshold $k$5, chosen within a stability plateau. Claude Sonnet 4.6 generated rubric-aligned assignment text, simulated Python OOP responses for synthetic student profiles, and scored those responses into a 24-element skill vector with $k$6 denoting “not applicable,” alongside a scalar score computed as the rounded mean of applicable skills times $k$7. The sample comprised 150 synthetic student profiles, 862 assignment records, and 7,788 paired skill-level observations across 23 tested skills.

Empirically, pooled Pearson agreement was $k$8 with $k$9 CI $k=24$0, implying $k=24$1, described as recovery of “roughly half the intended variance.” Mean bias was $k=24$2 with $k=24$3 CI $k=24$4, corresponding to approximately $k=24$5 points on the scalar score. Proficiency-level exact match was $k=24$6, and agreement within $k=24$7 adjacent level was $k=24$8. Agreement was strong for syntactically verifiable skills such as S05 Setter with Validation ($k=24$9), S09 Class Variable ($G:\boldsymbol{\theta}\mapsto x$0), S23 Raise Exception ($G:\boldsymbol{\theta}\mapsto x$1), S11 Composition ($G:\boldsymbol{\theta}\mapsto x$2), and S24 Try/Except ($G:\boldsymbol{\theta}\mapsto x$3), but near zero for design-level skills such as S19 ABC Definition ($G:\boldsymbol{\theta}\mapsto x$4), S17 Override Replace ($G:\boldsymbol{\theta}\mapsto x$5), S21 Polymorphism ($G:\boldsymbol{\theta}\mapsto x$6), and S20 Concrete Subclass, which showed degenerate scoring. The calibration curve exhibited low-skill overestimation, with true skill near $G:\boldsymbol{\theta}\mapsto x$7 scored near $G:\boldsymbol{\theta}\mapsto x$8, convergence near high skill, and mid-range compression with larger variance.

These properties mattered operationally because upward bias inflated Stage 1 scores for weak students near the routing threshold. Under the normal-error model

$G:\boldsymbol{\theta}\mapsto x$9

positive $E:x\mapsto \hat{\boldsymbol{\theta}}$0 and $E:x\mapsto \hat{\boldsymbol{\theta}}$1 increase false-upward routing below threshold. The paper’s sensitivity sweep reported a stability plateau for $E:x\mapsto \hat{\boldsymbol{\theta}}$2 with fewer than $E:x\mapsto \hat{\boldsymbol{\theta}}$3 routing flips, while misclassification declined from $E:x\mapsto \hat{\boldsymbol{\theta}}$4 at $E:x\mapsto \hat{\boldsymbol{\theta}}$5 to $E:x\mapsto \hat{\boldsymbol{\theta}}$6 at $E:x\mapsto \hat{\boldsymbol{\theta}}$7. A scaling comparison further showed that Claude Haiku 4.5 had signed bias $E:x\mapsto \hat{\boldsymbol{\theta}}$8 versus $E:x\mapsto \hat{\boldsymbol{\theta}}$9 for Sonnet 4.6, record-level $E\circ G \approx I$0 versus $E\circ G \approx I$1, and pooled skill-level $E\circ G \approx I$2 versus $E\circ G \approx I$3.

The principal proposed mechanism for strengthening InterGenEval is the use of granular, skill-decomposed rubrics that force both generation and evaluation through shared intermediate criteria. Complementary mitigations include linear score calibration via

$E\circ G \approx I$4

cross-model scoring, human-in-the-loop adjudication, external gold standards, programmatic static checks, adversarial validation, threshold buffering, and skill-level monitoring dashboards.

3. InterGenEval as an interaction-aware video-generation protocol

In "MATRIX: Mask Track Alignment for Interaction-aware Video Generation" (Jin et al., 8 Oct 2025), InterGenEval is defined as a protocol for evaluating interaction-aware text-conditioned video generation. Its explicit target is whether a model correctly realizes “who does what to whom,” grounds the correct subject and object, preserves those bindings over time, and avoids drift and hallucination. The protocol is designed for multi-instance interactions, including same-class entities, and for subject–object relations whose correctness cannot be captured by global CLIP/BLEU-style alignment alone.

The workflow begins from an image prompt and textual prompt specifying instances, their appearance descriptors, and one or more key interactions. Generated videos are preprocessed by extracting instance bounding boxes with SAM2. Evaluation frames include the first and last frames plus uniformly sampled intermediates with stride $E\circ G \approx I$5, and each instance is visualized with a distinct box color so that a multimodal evaluator can disambiguously associate language with visual entities. GPT-5 identifies key interactions and participating instances, then generates ten yes/no questions per interaction: six for Key Interaction Semantic Alignment (KISA), covering pre-, during-, and post-interaction states, and four for Semantic Grounding Integrity (SGI), checking whether actor and recipient roles are correctly grounded.

The raw scores are

$E\circ G \approx I$6

Temporal consistency is handled by Semantic Propagation Integrity (SPI), which penalizes emergence and disappearance across sampled frames. With emergence fraction $E\circ G \approx I$7, disappearance fraction $E\circ G \approx I$8, and penalty weight $E\circ G \approx I$9,

$\boldsymbol{\theta}$0

The reweighted metrics are

$\boldsymbol{\theta}$1

and the final Interaction Fidelity score is

$\boldsymbol{\theta}$2

When a prompt contains multiple interactions, KISA, SGI, SPI, and IF are computed per interaction and then averaged.

The reported evaluation sets comprise 60 synthetic and 58 real image–prompt pairs. Video generation is standardized at resolution $\boldsymbol{\theta}$3, length 49 frames, and one sample per prompt. Interaction categories include contact, force, transport, manipulation, and social interactions, spanning human–object, human–human, human–animal, and human–nature settings. On the main benchmark, IF was $\boldsymbol{\theta}$4 for CogVideoX-2B-I2V, $\boldsymbol{\theta}$5 for CogVideoX-5B-I2V, $\boldsymbol{\theta}$6 for Open-Sora-11B-I2V, $\boldsymbol{\theta}$7 for TaVid, and $\boldsymbol{\theta}$8 for MATRIX. The improvement of MATRIX over the best baseline, TaVid, was $\boldsymbol{\theta}$9 in KISA, $$\mathbb{E}\big[\hat{\boldsymbol{\theta}} \mid x \sim G(\boldsymbol{\theta})\big] \approx \boldsymbol{\theta}.$$0 in SGI, and $$\mathbb{E}\big[\hat{\boldsymbol{\theta}} \mid x \sim G(\boldsymbol{\theta})\big] \approx \boldsymbol{\theta}.$$1 in IF. A 2AFC human evaluation with 31 participants reported win rates above $$\mathbb{E}\big[\hat{\boldsymbol{\theta}} \mid x \sim G(\boldsymbol{\theta})\big] \approx \boldsymbol{\theta}.$$2 for MATRIX across all criteria.

The protocol is intentionally reference-grounded and propagation-sensitive. Its stated limitations are dependence on GPT-5 for question generation and answering, SPI sensitivity to small or occluded instances, and the coarser nature of bounding-box grounding relative to masks.

4. Standardized InterGenEval-style frameworks in other generative domains

Several other papers use InterGenEval more as a design pattern than as a fixed metric name: an evaluation layer should specify computation spaces, hyperparameters, and biologically or semantically meaningful endpoints explicitly, so that benchmarking is reproducible and auditable.

In single-cell generative modeling, "A Standardized Framework For Evaluating Gene Expression Generative Models" introduces the Generated Genetic Expression Evaluator (GGE), an open-source Python framework that exposes explicit computation spaces—raw, PCA, and DEG-restricted—alongside transparent implementations of Wasserstein, MMD, Energy distance, FID-like scores, coverage/precision, classifier two-sample tests, and perturbation-effect correlation, all computed per biological condition. The paper’s sensitivity analysis shows that changing only the space can alter $$\mathbb{E}\big[\hat{\boldsymbol{\theta}} \mid x \sim G(\boldsymbol{\theta})\big] \approx \boldsymbol{\theta}.$$3 from approximately $$\mathbb{E}\big[\hat{\boldsymbol{\theta}} \mid x \sim G(\boldsymbol{\theta})\big] \approx \boldsymbol{\theta}.$$4 in raw $$\mathbb{E}\big[\hat{\boldsymbol{\theta}} \mid x \sim G(\boldsymbol{\theta})\big] \approx \boldsymbol{\theta}.$$5 space to $$\mathbb{E}\big[\hat{\boldsymbol{\theta}} \mid x \sim G(\boldsymbol{\theta})\big] \approx \boldsymbol{\theta}.$$6 in PCA-50 and $$\mathbb{E}\big[\hat{\boldsymbol{\theta}} \mid x \sim G(\boldsymbol{\theta})\big] \approx \boldsymbol{\theta}.$$7 in PCA-25, with Energy distance dropping from approximately $$\mathbb{E}\big[\hat{\boldsymbol{\theta}} \mid x \sim G(\boldsymbol{\theta})\big] \approx \boldsymbol{\theta}.$$8 to approximately $$\mathbb{E}\big[\hat{\boldsymbol{\theta}} \mid x \sim G(\boldsymbol{\theta})\big] \approx \boldsymbol{\theta}.$$9. This is presented as evidence that the same metric name is otherwise uninterpretable without explicit declaration of space and hyperparameters (Rubbi et al., 11 Mar 2026).

In generative information retrieval, two complementary chapters argue that evaluation must separate end-to-end answer quality from component-level retrieval quality, and that LLM judging should be treated either as “slow search” or as a high-cost evaluator still anchored to human assessment. The proposed toolkit includes binary and graded relevance, DCG/NDCG, AP/MAP, MRR, pairwise Bradley–Terry modeling, subtopic or nugget coverage, calibration diagnostics such as Brier score and ECE, and agreement measures such as Cohen’s $r_i=\mathrm{corr}(\theta_i,\hat{\theta}_i)$0 and Krippendorff’s $r_i=\mathrm{corr}(\theta_i,\hat{\theta}_i)$1. One paper emphasizes the continuing need to ground evaluation in human assessment despite apparent circularity when LLMs evaluate LLM outputs (Alaofi et al., 2024), while another shows that pairwise preferences give the fewest ties and disagreements against human-derived categories, and that subtopic coverage provides a strong autonomy–auditability compromise (Arabzadeh et al., 2024).

In multilingual code generation, "IndicEval-XL: Bridging Linguistic Diversity in Code Generation Across Indic Languages" frames an InterGenEval-style benchmark around semantic rather than execution-only evaluation. The benchmark spans six Indic languages, twelve programming languages, and 6,720 function-level problems. Its translation-quality pipeline combines back-translation, BLEU and METEOR thresholds, a BERTScore filter requiring similarity at least $r_i=\mathrm{corr}(\theta_i,\hat{\theta}_i)$2, and manual review. Primary evaluation uses CodeBERTScore, while pass@k is deliberately de-emphasized because strict harness assumptions can penalize semantically correct code through function-name mismatches or alternate valid implementations (Singh et al., 26 Feb 2025).

A plausible implication of these frameworks is that InterGenEval, in this broader sense, denotes standardization through explicit decomposition: the evaluated system is not judged only by a holistic scalar, but by declared spaces, conditions, aspects, or constraints that make metric values interpretable.

5. Object-focused, interleaved, and aspect-taxonomic extensions

In image and interleaved multimodal generation, InterGenEval-style ideas are instantiated through instance-level verification, list-based sequence comparison, and hierarchical aspect taxonomies.

"GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment" evaluates compositional image properties such as object co-occurrence, position, count, and color. It uses Mask2Former instance segmentation, centroid-based spatial tests with offset parameter $r_i=\mathrm{corr}(\theta_i,\hat{\theta}_i)$3, and CLIP ViT-L/14 for masked-crop color classification. The main benchmark contains 553 prompts across six tasks, and agreement with humans is reported as $r_i=\mathrm{corr}(\theta_i,\hat{\theta}_i)$4 overall versus interannotator agreement of $r_i=\mathrm{corr}(\theta_i,\hat{\theta}_i)$5; on unanimous human judgments, agreement rises to $r_i=\mathrm{corr}(\theta_i,\hat{\theta}_i)$6. The paper explicitly motivates instance-level evaluation because FID and CLIPScore are holistic signals that often fail on counting, position, and attribute binding (Ghosh et al., 2023).

For interleaved image–text outputs in open-domain QA, "RAG-IGBench: Innovative Evaluation for RAG-based Interleaved Generation in Open-domain Question Answering" defines a reference-grounded metric suite over markdown answers that interleave text with image indices selected from a retrieved pool. The benchmark contains 6,057 queries, with average 12.96 retrieved images per query and average 2.26 generated images per answer. Text quality is measured primarily by ROUGE-1; image quality is measured by a normalized EditDistance over generated versus ground-truth image index sequences and a KendallScore over the order of the correctly selected intersection; image–text consistency is measured by CLIPScore and a contextual AlignmentScore. The overall score is the unweighted mean of ROUGE-1, EditDistance, KendallScore, AlignmentScore, and CLIPScore. On the main comparison, GPT-4o achieved the best composite Mean at 46.01, while the best open-source model, Qwen2.5VL-72B, reached 37.79. For metric validation on 200 sampled queries, the proposed image-quality metrics and overall composite correlated better with human judgments than FID, IS, or a GPT-4o-based judge (Zhang et al., 11 Oct 2025).

"FRABench and GenEval: Scaling Fine-Grained Aspect Evaluation across Tasks, Modalities" generalizes this logic into a hierarchical taxonomy of 112 aspects spanning Natural Language Generation, Image Understanding, Image Generation, and Interleaved Text-and-Image Generation. FRABench contains 60.4k pairwise samples and 325k aspect-level labels, with interleaved-specific universal aspects such as Text-Image Relationship, Image Coherence, and multiple complement families, and task-specific aspects such as Character Consistency, Action Consistency, Scene Consistency, and Semantic Consistency. The evaluator, trained from Qwen2-VL-7B-Instruct, uses aspect-specific prompts that produce feedback and 1–5 scores for each response. On human-annotated in-domain ITIG subsets, the paper reports that the trained evaluator reaches 70.0% agreement on universal aspects and 73.9% on task-specific aspects, slightly exceeding GPT-4o on those splits. Unified training across tasks and modalities is preferred over staged training because staged orders consistently suffer catastrophic forgetting (Hong et al., 19 May 2025).

Taken together, these works formalize a progression from object-centric verification, to interleaved sequence scoring, to fully taxonomic evaluation. This suggests that one of the most stable meanings of InterGenEval outside adaptive assessment is evaluation by decomposed multimodal constraints rather than by a single undifferentiated preference signal.

6. Multi-generational InterGenEval in Geometric Semantic Genetic Programming

A distinct usage appears in "The Effect of Multi-Generational Selection in Geometric Semantic Genetic Programming" (Castelli et al., 2022), where InterGenEval denotes a multi-generational selection scheme rather than a scoring protocol. The key enabling condition is the fast implementation of GSGP: because each individual stores pointers to parents in earlier populations together with random trees used by geometric semantic operators, the algorithm already retains the full evolutionary history and cached semantics. Older generations are therefore available for tournament selection at essentially zero extra memory cost.

In GSGP for symbolic regression, the semantics of a program $r_i=\mathrm{corr}(\theta_i,\hat{\theta}_i)$7 on training samples $r_i=\mathrm{corr}(\theta_i,\hat{\theta}_i)$8 is

$r_i=\mathrm{corr}(\theta_i,\hat{\theta}_i)$9

With Euclidean distance on semantics, crossover and mutation are geometric. The crossover offspring is

$r$0

with $r$1, and mutation is

$r$2

InterGenEval modifies only the source of tournament candidates. Under a uniform window over the last $r$3 generations,

$r$4

whereas under geometric decay,

$r$5

and generation $r$6 is sampled with probability $r$7.

The experiments use six regression benchmarks, 100 runs per dataset, population size 100, 100 generations, tournament size 4, crossover rate 0.9, mutation rate 0.3, and mutation step $r$8, with RMSE as fitness. Moderate historical reuse improved generalization on most benchmarks. The uniform window U5 and geometric decay G0.25 were reported as consistently improving test RMSE over standard GSGP. Example median test RMSEs include airfoil self-noise, where GSGP scored 34.44, U5 31.83, and G0.25 32.93; yacht hydrodynamics, where GSGP scored 13.55, U5 12.99, and G0.25 13.30; and LD50 toxicity, where GSGP scored 2262.15 and G0.25 2216.47. By contrast, larger windows such as U50 and U100, and heavier recency bias such as G0.75, tended to degrade performance. The %PPB dataset exhibited overfitting across methods.

The paper’s explanation is geometric: admitting parents from earlier generations expands the effective convex hull of reachable semantics and delays premature shrinkage of the search space. Here InterGenEval is therefore not an evaluator of outputs in the usual benchmark sense, but a way of exploiting inter-generational structure already stored by the algorithm.

A plausible unifying interpretation across all usages is that InterGenEval names procedures that make latent structure explicit—skill vectors, interaction bindings, biological conditions, subtopics, aspect trees, or archived generations—and then test whether that structure is preserved, recovered, or usefully reintroduced. The term’s polysemy is therefore substantive rather than accidental: each usage addresses a different failure mode of generative systems, but all do so by replacing holistic judgment with structured, inspectable criteria.