GenEvolve-Data: Process-Level Image Generation

• GenEvolve-Data is a comprehensive dataset featuring multi-turn tool trajectories, selected references, and internal reasoning for process-level image generation.
• It enables both supervised cold start and self-evolution training by contrasting best-worst trajectories under a structured reward protocol.
• The dataset employs recipe-controlled prompt generation and strict filtering to ensure high-quality evidence acquisition and transparent tool orchestration.

GenEvolve-Data is the training and construction substrate for GenEvolve, a “self-evolving image generation agent” that is trained as a tool-orchestrated visual trajectory learner, rather than as a simple prompt rewriter. It is designed to “go beyond ordinary prompt-rewriting corpora” by providing complete generation pipelines in which an agent must acquire missing evidence, retrieve and select visual references, activate internal generation knowledge, and synthesize a generator-facing prompt-reference program $z=(g,R)$, where $g$ is the final prompt and $R$ is a small ordered set of selected reference images. In this formulation, GenEvolve-Data makes the whole process of image generation observable and trainable: not just the final prompt, but also search behavior, reference choice, skill routing, and the final program consumed by a downstream image generator (Chen et al., 20 May 2026).

1. Position within the GenEvolve framework

GenEvolve-Data is tied directly to the claim that open-ended image generation is no longer a one-shot text-to-image problem. The underlying framework argues that high-quality generation often requires deciding when to search for facts, which retrieved images to trust, when to invoke built-in generation skills such as text rendering or spatial layout, and how to bind all of that into a prompt-reference program. GenEvolve-Data is therefore constructed to support three functions: supervised tool trajectories for cold start, filtered GT image cases that support self-evolution via reward feedback, and a held-out benchmark split for evaluation (Chen et al., 20 May 2026).

A central design point is that the agent does not directly generate an image. Instead, it operates over a trajectory formalized as

$\tau=(x,a_1,o_1,\ldots,a_T,o_T,z,\hat{y},r,d),$

where $x$ is the user request, $z$ is the prompt-reference program, $\hat{y}=G(g,R)$ is the generated image, $r$ is reward, and $d$ are diagnostics. This makes the entire generation process trainable, not just the final prompt.

The dataset is explicitly distinguished from ordinary prompt datasets and prompt-rewriting corpora. It stores multi-turn tool trajectories, selected references, and tool reasoning; it separates Knowledge-Anchored and Quality-Anchored tracks; it includes filtered GT image cases for reward-based self-evolution; and it constructs best-worst contrastive visual experience, not just positive examples. A plausible implication is that GenEvolve-Data is intended as a process-level supervision resource rather than as a static prompt-output corpus.

2. Data schema and trajectory representation

A supervised example in GenEvolve-Data contains a user request, a multi-turn tool trajectory, and a final program: $$\tau^{\star}=(a_1,o_1,\ldots,a_T,o_T,z^{\star}), \qquad z^{\star}=(g^{\star},R^{\star}).$$ The trajectory includes the assistant’s reasoning/tool calls and the observations returned by tools. The tool calls come from three families: $g$0, $g$1, and $g$2 (Chen et al., 20 May 2026).

The final answer is a structured JSON-like prompt-reference program. In the appendix schema, it can contain gen_prompt, reference_images, selected_skills, and rationale. The generated prompt must refer to references by ordinal phrase only, for example “the first reference image,” rather than by raw image IDs or URLs. The paper also stresses that the selected skills are not passed as hidden controls to the generator; they are tracked for supervision, diagnostics, and experience construction. This matters because a common misconception is to treat the skill field as an inference-time control channel. In the reported schema, its role is supervisory and diagnostic rather than hidden conditioning.

The tool order is intentionally not fixed. Knowledge-heavy cases may search facts first; reference-sensitive cases may begin with image search; and quality-anchored cases may invoke generation knowledge such as text rendering or spatial layout. This suggests that the dataset supervises adaptive orchestration rather than a single canonical action sequence.

3. Prompt construction and task taxonomy

GenEvolve-Data is not collected from unconstrained prompts. Its construction begins with recipe-controlled prompt generation. Each recipe specifies task family, factual gap, visual anchor, dominant visible requirement, optional secondary constraints, and difficulty. These labels are used for coverage control and stratified splitting, but they are not exposed to the agent during training (Chen et al., 20 May 2026).

The prompt pool covers two complementary tracks. Knowledge-Anchored prompts require external grounding for entities, events, places, objects, or visual facts. Quality-Anchored prompts emphasize quality-sensitive generation requirements such as text layout, spatial composition, counting, anatomy, material consistency, aesthetics, and creative transfer. The distinction is operational rather than merely descriptive: all prompts require image search, while text search is used for all Knowledge-Anchored prompts plus 1,973 Quality-Anchored prompts.

The appendix reports 19,990 valid prompts after deduplication, consisting of 11,999 Knowledge-Anchored and 7,991 Quality-Anchored prompts, with average prompt length about 65 words. Difficulty is reported as 13,333 hard, 6,654 medium, and 3 easy. The prompt categories are:

• Architecture
• city streets
• public figures
• products
• vehicles
• events
• science
• artifacts
• text/layout
• spatial
• counting
• anatomy
• attributes
• material
• aesthetics
• creative transfer

These categories indicate that the dataset is organized around generation demands rather than around a single domain ontology. A plausible implication is that the construction logic is intended to induce broad coverage of both factual grounding and generator-sensitive visual control.

4. Construction pipeline, filtering, and corpus statistics

Teacher trajectories are produced with Seed2.0 and Gemini 3 Pro, which emit reasoning and then one tool call or final answer per turn. For self-evolution, multiple rollouts are sampled for the same prompt, and GenEvolve identifies the best and worst trajectories for that prompt: $g$3 If the reward gap exceeds a threshold, the best-worst pair is distilled into structured visual experience. The paper states that the best-worst pair is summarized into five experience slots capturing search strategy, internal knowledge activation, reference selection, prompt-reference construction, and failure-avoidance lessons (Chen et al., 20 May 2026).

Before training, the dataset is aggressively filtered. Hard programmatic checks remove incomplete tool loops, invalid reference selections, raw URL or ID leakage, missing ordinal reference wording, underspecified final programs, missing image search, invalid reference counts, invalid local paths, unparseable JSON, invalid skill names, meaningless reasoning, and unsafe content. A VLM-based filter then checks prompt suitability, reference grounding, trajectory process quality, skill integration, final prompt faithfulness, and supervised training value.

The reported corpus statistics are as follows:

Component	Count	Notes
Valid prompts after deduplication	19,990	11,999 Knowledge-Anchored; 7,991 Quality-Anchored
Structurally valid trajectories kept	13,379 of 19,320	$g$4
Successful GT images	4,321 of 4,379	$g$5
Filtered GT image cases retained	3,175	$g$6

For GT image generation, the paper uses Nano Banana Pro to render high-quality images from the teacher’s prompt-reference programs and selected references. A second visual filter evaluates prompt compliance, reference utilization, visual coherence, and image quality. Pairs caused only by protocol failures, such as missing references, are discarded because they do not yield reusable strategy. This indicates that the retained data are intended to encode reusable process knowledge rather than accidental execution defects.

5. Training roles: supervised cold start and self-evolution

GenEvolve-Data supports two training stages. In the first stage, the supervised split is used to cold-start Qwen3-VL-8B-Instruct into a tool-orchestrated agent. The supervised view preserves the full tool-loop conversation and all images shown to the model, including candidate references, so that the student learns evidence acquisition, candidate comparison, reference selection, internal-knowledge activation, and prompt-reference program construction (Chen et al., 20 May 2026).

Only assistant-side tokens are optimized; user prompts and tool observations are masked from the loss. The appendix gives the supervised split as 8,800 training and 200 held-out trajectories. This suggests that the role of GenEvolve-Data in supervised learning is not to imitate a final text answer, but to teach the sequential control policy that produces a valid generation program.

In the second stage, the self-evolution split is used to mine best-worst comparisons, build prompt-keyed experience, and train with GRPO plus SDL. The teacher view is patched with retrieved experience: $g$7 The student sees the normal inference context $g$8, while the teacher gets the same context augmented with visual experience. The paper characterizes this as a teacher-only experience injection regime for token-level self-distillation.

The appendix reports the released settings as $g$9, $R$0, experience buffer capacity = 500 bundles, and minimum reward gap $R$1. It also states that only the top 10% of special tokens in each sequence by $R$2 are kept, concentrating teacher signal on the decision tokens where the experience-conditioned teacher and student disagree most strongly. This suggests that GenEvolve-Data is used not only as a trajectory corpus but also as a mechanism for extracting sparse, high-value supervisory signals from trajectory contrasts.

6. Evaluation split, benchmark relation, and illustrative cases

GenEvolve-Data is primarily a training substrate, but it also includes a held-out evaluation split. GenEvolve-Bench is the diagnostic benchmark carved out from the same construction pipeline, with no exact overlap with the self-evolution training pool. The benchmark evaluates open-ended image-generation agents under a unified KScore protocol and covers both Knowledge-Anchored and Quality-Anchored prompts (Chen et al., 20 May 2026).

The appendix states that the current self-evolution/eval image-case split uses 3,175 filtered GT image cases, divided into a 2,575-case self-evolution training pool and about 600-case evaluation pool. Of the training pool, 2,446 are optimization cases and 129 are internal validation cases. Benchmark metadata records category, difficulty, search flags, expected reference targets, and quality-requirement tags.

The appendix also includes three detailed cases that clarify what GenEvolve-Data records. In the Crucible Theatre / World Snooker Championship case, the best trajectory searched for the winner first, found Luca Brecel, and then used the Belgian flag stripes correctly, whereas the worst trajectory added “flag” to the search query and grounded the final prompt on the wrong flag colors. In the Aerotrain I80 travel-poster case, both trajectories found 430.4 km/h (267 mph), set in 1974, but the better trajectory also called text_rendering, decomposing the poster text into separate lines and placing them in the lower third, whereas the worse one produced garbled typography. In the Hundertwasser / Piet Blom side-by-side buildings case, the better trajectory used image search for both buildings and then called spatial_layout, text_rendering, and physical_material_consistency, while the worse trajectory produced merged buildings and sign failures.

These cases support an important clarification. GenEvolve-Data is not merely a dataset of retrieved references, and it is not merely a dataset of prompts paired with outputs. It records actionable decisions about when to search, which skill to query, how to assign reference roles, and how to write the final program so the generator can obey it.

7. Conceptual significance and boundaries

The paper’s central claim is that GenEvolve-Data provides “tool-orchestrated trajectories that teach the agent how to acquire external evidence, activate internal generation knowledge, and construct prompt-reference programs,” together with “filtered GT image cases that make visual feedback meaningful for self-evolution” (Chen et al., 20 May 2026). Within the GenEvolve framework, it is therefore both a supervision corpus and an experience-construction substrate.

Several boundaries are explicit. The self-evolution view removes the teacher trajectory and teacher final program; it contains only the raw prompt, GT image path, and metadata for reward evaluation. The selected skills are not hidden generator controls. The prompt pool is recipe-controlled rather than unconstrained. Quality control excludes cases that do not yield reusable strategy, such as protocol failures. These design choices indicate that the dataset is curated for controllable process learning rather than for unconstrained web-scale collection.

A common misunderstanding would be to reduce GenEvolve-Data to a prompt-engineering dataset. The construction details suggest a narrower and more technical interpretation: it is a trajectory-centric dataset for training an image-generation agent that must orchestrate tools, references, and internal knowledge under open-ended requests. Another plausible implication is that its distinctive contribution lies in making agentic generation behavior legible at the level of search, routing, and program synthesis, not only at the level of final image preference.