---
title: 'GenEvolve-Data: Process-Level Image Generation'
url: https://www.emergentmind.com/topics/genevolve-data
type: topic
---

# GenEvolve-Data: Process-Level Image Generation

GenEvolve-Data is the training and construction substrate for GenEvolve, a “self-evolving image generation agent” that is trained as a **tool-orchestrated visual trajectory learner**, rather than as a simple prompt rewriter. It is designed to “go beyond ordinary prompt-rewriting corpora” by providing **complete generation pipelines** in which an agent must acquire missing evidence, retrieve and select visual references, activate internal generation knowledge, and synthesize a generator-facing **prompt-reference program** \(z=(g,R)\), where \(g\) is the final prompt and \(R\) is a small ordered set of selected reference images. In this formulation, GenEvolve-Data makes the whole process of image generation observable and trainable: not just the final prompt, but also search behavior, reference choice, skill routing, and the final program consumed by a downstream image generator [2605.21605].

## 1. Position within the GenEvolve framework

GenEvolve-Data is tied directly to the claim that open-ended image generation is no longer a one-shot text-to-image problem. The underlying framework argues that high-quality generation often requires deciding when to search for facts, which retrieved images to trust, when to invoke built-in generation skills such as text rendering or spatial layout, and how to bind all of that into a prompt-reference program. GenEvolve-Data is therefore constructed to support three functions: **supervised tool trajectories for cold start**, **filtered GT image cases that support self-evolution via reward feedback**, and **a held-out benchmark split for evaluation** [2605.21605].

A central design point is that the agent does not directly generate an image. Instead, it operates over a trajectory formalized as
\[
\tau=(x,a_1,o_1,\ldots,a_T,o_T,z,\hat{y},r,d),
\]
where \(x\) is the user request, \(z\) is the prompt-reference program, \(\hat{y}=G(g,R)\) is the generated image, \(r\) is reward, and \(d\) are diagnostics. This makes the entire generation process trainable, not just the final prompt.

The dataset is explicitly distinguished from ordinary prompt datasets and prompt-rewriting corpora. It stores **multi-turn tool trajectories**, **selected references**, and **tool reasoning**; it separates **Knowledge-Anchored** and **Quality-Anchored** tracks; it includes **filtered GT image cases** for reward-based self-evolution; and it constructs **best-worst contrastive visual experience**, not just positive examples. A plausible implication is that GenEvolve-Data is intended as a process-level supervision resource rather than as a static prompt-output corpus.

## 2. Data schema and trajectory representation

A supervised example in GenEvolve-Data contains a **user request**, a **multi-turn tool trajectory**, and a **final program**:
\[
\tau^{\star}=(a_1,o_1,\ldots,a_T,o_T,z^{\star}), \qquad z^{\star}=(g^{\star},R^{\star}).
\]
The trajectory includes the assistant’s reasoning/tool calls and the observations returned by tools. The tool calls come from three families: \(\texttt{search}(q)\), \(\texttt{image\_search}(q)\), and \(\texttt{query\_knowledge}(\text{skill\_name})\) [2605.21605].

The final answer is a structured JSON-like prompt-reference program. In the appendix schema, it can contain `gen_prompt`, `reference_images`, `selected_skills`, and `rationale`. The generated prompt must refer to references by **ordinal phrase** only, for example “the first reference image,” rather than by raw image IDs or URLs. The paper also stresses that the selected skills are **not passed as hidden controls to the generator**; they are tracked for supervision, diagnostics, and experience construction. This matters because a common misconception is to treat the skill field as an inference-time control channel. In the reported schema, its role is supervisory and diagnostic rather than hidden conditioning.

The tool order is intentionally **not fixed**. Knowledge-heavy cases may search facts first; reference-sensitive cases may begin with image search; and quality-anchored cases may invoke generation knowledge such as text rendering or spatial layout. This suggests that the dataset supervises adaptive orchestration rather than a single canonical action sequence.

## 3. Prompt construction and task taxonomy

GenEvolve-Data is not collected from unconstrained prompts. Its construction begins with **recipe-controlled prompt generation**. Each recipe specifies **task family**, **factual gap**, **visual anchor**, **dominant visible requirement**, **optional secondary constraints**, and **difficulty**. These labels are used for coverage control and stratified splitting, but they are not exposed to the agent during training [2605.21605].

The prompt pool covers two complementary tracks. **Knowledge-Anchored** prompts require external grounding for entities, events, places, objects, or visual facts. **Quality-Anchored** prompts emphasize quality-sensitive generation requirements such as text layout, spatial composition, counting, anatomy, material consistency, aesthetics, and creative transfer. The distinction is operational rather than merely descriptive: all prompts require image search, while text search is used for all Knowledge-Anchored prompts plus **1,973** Quality-Anchored prompts.

The appendix reports **19,990 valid prompts after deduplication**, consisting of **11,999 Knowledge-Anchored** and **7,991 Quality-Anchored** prompts, with average prompt length about **65 words**. Difficulty is reported as **13,333 hard**, **6,654 medium**, and **3 easy**. The prompt categories are:

- Architecture  
- city streets  
- public figures  
- products  
- vehicles  
- events  
- science  
- artifacts  
- text/layout  
- spatial  
- counting  
- anatomy  
- attributes  
- material  
- aesthetics  
- creative transfer  

These categories indicate that the dataset is organized around generation demands rather than around a single domain ontology. A plausible implication is that the construction logic is intended to induce broad coverage of both factual grounding and generator-sensitive visual control.

## 4. Construction pipeline, filtering, and corpus statistics

Teacher trajectories are produced with **Seed2.0** and **Gemini 3 Pro**, which emit reasoning and then one tool call or final answer per turn. For self-evolution, multiple rollouts are sampled for the same prompt, and GenEvolve identifies the best and worst trajectories for that prompt:
\[
\tau^+=\arg\max_i R(\tau_i),\qquad \tau^-=\arg\min_i R(\tau_i),\qquad \Delta=R(\tau^+)-R(\tau^-).
\]
If the reward gap exceeds a threshold, the best-worst pair is distilled into structured visual experience. The paper states that the best-worst pair is summarized into five experience slots capturing **search strategy**, **internal knowledge activation**, **reference selection**, **prompt-reference construction**, and **failure-avoidance lessons** [2605.21605].

Before training, the dataset is aggressively filtered. Hard programmatic checks remove incomplete tool loops, invalid reference selections, raw URL or ID leakage, missing ordinal reference wording, underspecified final programs, missing image search, invalid reference counts, invalid local paths, unparseable JSON, invalid skill names, meaningless reasoning, and unsafe content. A VLM-based filter then checks prompt suitability, reference grounding, trajectory process quality, skill integration, final prompt faithfulness, and supervised training value.

The reported corpus statistics are as follows:

| Component | Count | Notes |
|---|---:|---|
| Valid prompts after deduplication | 19,990 | 11,999 Knowledge-Anchored; 7,991 Quality-Anchored |
| Structurally valid trajectories kept | 13,379 of 19,320 | \(69.2\%\) |
| Successful GT images | 4,321 of 4,379 | \(98.7\%\) |
| Filtered GT image cases retained | 3,175 | \(73.5\%\) |

For GT image generation, the paper uses **Nano Banana Pro** to render high-quality images from the teacher’s prompt-reference programs and selected references. A second visual filter evaluates prompt compliance, reference utilization, visual coherence, and image quality. Pairs caused only by protocol failures, such as missing references, are discarded because they do not yield reusable strategy. This indicates that the retained data are intended to encode reusable process knowledge rather than accidental execution defects.

## 5. Training roles: supervised cold start and self-evolution

GenEvolve-Data supports two training stages. In the first stage, the supervised split is used to cold-start **Qwen3-VL-8B-Instruct** into a tool-orchestrated agent. The supervised view preserves the full tool-loop conversation and all images shown to the model, including candidate references, so that the student learns evidence acquisition, candidate comparison, reference selection, internal-knowledge activation, and prompt-reference program construction [2605.21605].

Only assistant-side tokens are optimized; user prompts and tool observations are masked from the loss. The appendix gives the supervised split as **8,800 training** and **200 held-out trajectories**. This suggests that the role of GenEvolve-Data in supervised learning is not to imitate a final text answer, but to teach the sequential control policy that produces a valid generation program.

In the second stage, the self-evolution split is used to mine best-worst comparisons, build prompt-keyed experience, and train with **GRPO plus SDL**. The teacher view is patched with retrieved experience:
\[
c_E(x)=\operatorname{Patch}(c(x),M_x).
\]
The student sees the normal inference context \(c(x)\), while the teacher gets the same context augmented with visual experience. The paper characterizes this as a teacher-only experience injection regime for token-level self-distillation.

The appendix reports the released settings as \(\lambda_{\text{SDL}}=2.0\), \(\rho_{\max}=2.0\), **experience buffer capacity = 500 bundles**, and **minimum reward gap \(\delta_{\min}=0.20\)**. It also states that only the top **10%** of special tokens in each sequence by \(|\log\pi^E_\theta-\log\pi^S_\theta|\) are kept, concentrating teacher signal on the decision tokens where the experience-conditioned teacher and student disagree most strongly. This suggests that GenEvolve-Data is used not only as a trajectory corpus but also as a mechanism for extracting sparse, high-value supervisory signals from trajectory contrasts.

## 6. Evaluation split, benchmark relation, and illustrative cases

GenEvolve-Data is primarily a **training substrate**, but it also includes a held-out evaluation split. **GenEvolve-Bench** is the **diagnostic benchmark** carved out from the same construction pipeline, with no exact overlap with the self-evolution training pool. The benchmark evaluates open-ended image-generation agents under a unified **KScore** protocol and covers both **Knowledge-Anchored** and **Quality-Anchored** prompts [2605.21605].

The appendix states that the current self-evolution/eval image-case split uses **3,175 filtered GT image cases**, divided into a **2,575-case self-evolution training pool** and about **600-case evaluation pool**. Of the training pool, **2,446** are optimization cases and **129** are internal validation cases. Benchmark metadata records category, difficulty, search flags, expected reference targets, and quality-requirement tags.

The appendix also includes three detailed cases that clarify what GenEvolve-Data records. In the **Crucible Theatre / World Snooker Championship** case, the best trajectory searched for the winner first, found **Luca Brecel**, and then used the Belgian flag stripes correctly, whereas the worst trajectory added “flag” to the search query and grounded the final prompt on the wrong flag colors. In the **Aerotrain I80 travel-poster** case, both trajectories found **430.4 km/h (267 mph), set in 1974**, but the better trajectory also called **text_rendering**, decomposing the poster text into separate lines and placing them in the lower third, whereas the worse one produced garbled typography. In the **Hundertwasser / Piet Blom** side-by-side buildings case, the better trajectory used image search for both buildings and then called **spatial_layout**, **text_rendering**, and **physical_material_consistency**, while the worse trajectory produced merged buildings and sign failures.

These cases support an important clarification. GenEvolve-Data is not merely a dataset of retrieved references, and it is not merely a dataset of prompts paired with outputs. It records actionable decisions about when to search, which skill to query, how to assign reference roles, and how to write the final program so the generator can obey it.

## 7. Conceptual significance and boundaries

The paper’s central claim is that GenEvolve-Data provides “tool-orchestrated trajectories that teach the agent how to acquire external evidence, activate internal generation knowledge, and construct prompt-reference programs,” together with “filtered GT image cases that make visual feedback meaningful for self-evolution” [2605.21605]. Within the GenEvolve framework, it is therefore both a supervision corpus and an experience-construction substrate.

Several boundaries are explicit. The self-evolution view removes the teacher trajectory and teacher final program; it contains only the raw prompt, GT image path, and metadata for reward evaluation. The selected skills are not hidden generator controls. The prompt pool is recipe-controlled rather than unconstrained. Quality control excludes cases that do not yield reusable strategy, such as protocol failures. These design choices indicate that the dataset is curated for controllable process learning rather than for unconstrained web-scale collection.

A common misunderstanding would be to reduce GenEvolve-Data to a prompt-engineering dataset. The construction details suggest a narrower and more technical interpretation: it is a trajectory-centric dataset for training an image-generation agent that must orchestrate tools, references, and internal knowledge under open-ended requests. Another plausible implication is that its distinctive contribution lies in making agentic generation behavior legible at the level of search, routing, and program synthesis, not only at the level of final image preference.

Source: https://www.emergentmind.com/topics/genevolve-data