GenEvolve: Self-Evolving Image Generation Agents via Tool-Orchestrated Visual Experience Distillation

Abstract: Open-ended image generation is no longer a simple prompt-to-image problem. High-quality generation often requires an agent to combine a model's internal generative ability with external resources. As requests become more diverse and demanding, we aim to develop a general image-generation agent that can self-evolve through trajectories and use tools more effectively across varied generation challenges. To this end, we propose GenEvolve, a self-evolving framework based on Tool-Orchestrated Visual Experience Distillation. In GenEvolve, each generation attempt is modeled as a tool-orchestrated trajectory, where the agent gathers evidence, selects references, invokes generation skills, and composes them into a prompt-reference program. Unlike existing agentic generation methods that mainly rely on image-level scalar rewards, GenEvolve compares multiple trajectories for the same request and abstracts best-worst differences into structured visual experience, provided only to a privileged teacher branch. Inspired by on-policy self-distillation, Visual Experience Distillation provides dense token-level supervision, helping the student internalize better search, knowledge activation, reference selection, and prompt construction. We further construct GenEvolve-Data and GenEvolve-Bench. Experiments on public benchmarks and GenEvolve-Bench show substantial gains over strong baselines, achieving state-of-the-art performance among current image-generation frameworks. Our website is as follows: https://ephemeral182.github.io/GenEvolve/

• The paper presents a novel framework that recasts image generation as an agentic trajectory optimization problem for improved tool coordination and visual experience distillation.
• It integrates structured rollouts combining search, skill activation, and reference synthesis to refine the generation process.
• Experimental results demonstrate enhanced metrics such as KScore and WiScore, validating the framework’s impact on multimodal reasoning and transferability.

GenEvolve: Self-Evolving Image Generation Agents via Tool-Orchestrated Visual Experience Distillation

Reformulating Image Generation as Agentic Trajectory Optimization

The GenEvolve framework introduces a principled shift from traditional prompt-to-image paradigms toward agentic, tool-orchestrated visual trajectories. Unlike prior models that directly transform textual prompts into images or augment generation with passive retrieval, GenEvolve formalizes generation as an active process: an agent autonomously decides when to acquire external evidence, selects visual references, invokes internal generative knowledge (through callable skills), and synthesizes a prompt-reference program that downstream generators execute. Each generation attempt thus becomes a structured trajectory with multimodal decision points, tool interleaving, and programmatic orchestration.

Figure 1: GenEvolve agent structure: rollouts integrate search, reference retrieval, skill activation, and prompt synthesis with group-relative policy optimization and Visual Experience Self-Distillation.

This formulation directly addresses key weaknesses in prior agentic and generator-centric approaches, particularly the lack of explicit optimization over tool coordination, reference grounding, knowledge activation, and actionable feedback. By treating the entire visual trajectory—including tool calls and prompt construction—as the learnable object, GenEvolve occupies a distinctly compositional and interactive regime.

Tool-Orchestrated Trajectory Data and Diagnostics

GenEvolve-Data and GenEvolve-Bench constitute a comprehensive data substrate for training and evaluating agentic image-generation policies. GenEvolve-Data is constructed from stratified prompt pools, teacher agent rollouts, programmatic and vision-language filtering, and ground truth (GT) image generation. Prompts are categorized into Knowledge-Anchored (external grounding, factual evidence) and Quality-Anchored (visible quality challenges, spatial, text, material, anatomy) tracks. Teacher trajectories employ strong multimodal models (Seed2.0, Gemini 3 Pro), issuing search/image queries, skill activations, and synthesizing final programs.

Figure 2: Data pipeline overview—teacher trajectories for prompt solving, evidence acquisition, program construction, GT image generation, split stratification, and evaluation.

The resulting set precisely controls coverage and diagnostic value: trajectories are filtered for completeness, reference faithfulness, skill integration, and program compliance. GT images instantiated by Nano Banana Pro serve as strong visual targets, enabling effective self-evolution and actionable benchmarks. GenEvolve-Bench evaluates agentic policies across both external grounding and quality control, reporting image quality via KScore, faithfulness, correctness, text, and aesthetics.

Figure 3: Hierarchical prompt pool—Knowledge- and Quality-Anchored categories for stratified analysis.

Figure 4: Data construction statistics—filtering, GT generation, split stratification.

Visual Experience Distillation: Dense Agentic Supervision

The central innovation is Visual Experience Distillation (VED), which establishes a self-evolving loop through structured trajectory comparison. For each prompt, multiple agentic rollouts are sampled; the best and worst trajectories—determined by reward gap—are contrasted to abstract key lessons across five slots: search strategy, skill activation, reference selection, prompt synthesis, and failure avoidance.

Structured visual experience is provided only to a privileged teacher branch during distillation. The student receives plain inference context; the teacher incorporates prompt-keyed experience bundles extracted by VLM-based summarization and prompt embedding similarity. Token-level supervision employs a sampled-token reverse-KL importance-weighted objective, concentrating guidance on tokens where student and teacher diverge. This closes the loop: as the updated student policy yields better rollouts, subsequent VED provides denser, more actionable feedback.

Figure 5: Experience-conditioned distillation—token-level teacher guidance, opposing or supporting student decisions based on extracted bundles.

Self-evolution dynamics empirically confirm progressive reward improvement and policy convergence under VED; reward curves steadily rise, while SDL loss decreases as agent internalizes visual strategy and orchestration skills.

Figure 6: Training progression—reward and SDL loss trends indicate stable policy improvement and experience absorption.

Experimental Results and Empirical Analysis

Experiments span GenEvolve-Bench and the external WISE benchmark, testing both generator-native and agentic baselines with open and strong downstream generators (Qwen-Image-Edit, Nano Banana Pro). GenEvolve achieves substantial gains:

• With Qwen-Image-Edit, GenEvolve elevates KScore from 0.3493 (Gen-Searcher baseline) to 0.3663, with visual correctness rising from 0.1050 to 0.1338.
• Under Nano Banana Pro, GenEvolve attains 0.5739 KScore, surpassing both raw generator and Gen-Searcher with strong generator, demonstrating orchestration transferability rather than overfitting.

  Figure 7: Top: Qualitative agent generation across diverse open-ended requests; Bottom: Quantitative benchmark comparisons—GenEvolve outperforms SOTA generators and recent agentic baselines.

On the external WISE benchmark, GenEvolve achieves a WiScore of 0.82, outperforming all evaluated baselines, especially in chemistry (0.83) and biology (0.83)—categories where tool-orchestrated evidence acquisition confers maximal benefit.

Ablation studies confirm the necessity of each stage: base workflow, SFT cold start, scalar-reward GRPO, and VED all contribute, with token-level distillation providing complementary improvements over trajectory-level rewards. Failure mode analysis reveals that subtle search query errors, missed skill activation, and vague prompt construction propagate into severe generation failures. VED systematically extracts and distills actionable corrections—e.g., search query decomposition, skill routing, explicit spatial anchoring, and reference-role assignment.

Figure 8: Visual comparative benchmark cases—orange shows external knowledge requirements, blue marks internal skill constraints. GenEvolve enhances generation quality by enforcing agentic orchestration.

Practical and Theoretical Implications

GenEvolve demonstrates that image generation is not reducible to prompt rewriting or black-box generator invocation. The trajectory-level compositional agentic process is essential in addressing multimodal reasoning, factual grounding, reference utilization, and visible quality control. Practically, GenEvolve-style frameworks can serve as a general backbone for open-ended multimodal agents, supporting compositional skills, evidence integration, and generator-transferable orchestration. The use of structured visual experience as teacher-only privileged signal opens new directions in fine-grained self-distillation for both visual and multimodal agent policies.

Theoretically, the token-level sampled-KL importance-weighted objective in VED aligns with emerging paradigms in on-policy context distillation, skill-based agent learning, and group-relative policy optimization. Agentic trajectory learning expands the RLHF-style optimization scope, establishing dense credit assignment and actionable correction at the decision-token level.

Future research may focus on scaling bundle extraction, integrating more sophisticated VLM judges, expanding skill grammars, optimizing cross-generator transfer, and benchmarking compositional agent policies in broader multimodal reasoning and task planning domains.

Conclusion

GenEvolve advances image generation by combining tool orchestration, skill-activated agentic reasoning, and Visual Experience Distillation. Experimental results substantiate improved generation quality, robust orchestration transfer, and actionable failure correction across challenging open-ended requests. The framework validates trajectory-based optimization for multimodal agent policies, establishing a foundation for future compositional agents in visual and general multimodal domains (2605.21605).