- The paper introduces MemoGen, a training-free agent framework that stores positive strategies and negative warnings from prior text-to-image episodes while keeping the controller and image generator frozen.
- MemoGen combines intent planning, visual-text retrieval, web search, image inspection, and an independent multimodal memory judge to convert generation histories into reusable cross-task guidance.
- After two evolution rounds, MemoGen reaches 0.91 on WISE and 0.52 on Mind-Bench, outperforming reported proprietary and agentic baselines, although gains may depend on judge calibration and repeated-task evaluation.
Motivation and problem statement
MemoGen addresses a specific gap in agentic text-to-image (T2I) generation: existing retrieval-augmented and search-augmented systems—Gen-Searcher, Mind-Brush, T2I-Copilot, GenArtist—improve the current generation by acquiring external knowledge and references, but discard the resulting experience once the episode ends. Successful prompt strategies and failure modes are not systematically preserved or reused. The paper's central question is whether a T2I system can continually improve from its own generation history without updating the underlying generator. The authors' position is that many generation failures stem not from insufficient synthesis capability but from missing task understanding, external knowledge, and explicit visual constraints; consequently, improvement can occur entirely at the agent layer.
Method
MemoGen is a training-free framework in which an LLM/VLM controller manages each generation episode as an agent trajectory over tool calls. The underlying image generator (Qwen-Image / Qwen-Image-Edit-2509) and controller remain fully frozen across all evolution rounds. Four capability classes are exposed to the controller:
- Intent understanding and planning: a structured intent representation covering objects, attributes, spatial relations, temporal states, style requirements, negative constraints, and knowledge dependencies, followed by reference selection and a runtime-updated generation plan.
- Knowledge and reference acquisition: three complementary tools—a local visual-text RAG built on the WIT corpus, dynamic web/image search via Serper for long-tail entities, and find/grep-style retrieval over the experience memory pool.
- Generation and visual inspection: candidate images are inspected for visible gaps, with cropped or resized views for fine-grained checks, producing attempt-level internal feedback fi(r,k) with outcomes in {sat,unsat,uncertain}.
- Experience memory management: episodes are written back as structured records containing prompts, retrieved evidence, references, configurations, observations, feedback, a textual lesson, a polarity label ρi(r)∈{positive,negative}, and a retrieval policy (reuse_as_strategy vs. use_as_warning).
A distinctive design choice is the two-stage feedback memory. Internal visual feedback controls within-episode repair, while an independent multimodal memory judge Jmem produces a binary acceptance signal and structured diagnosis after episode completion, using five task-visible criteria (instruction faithfulness, intent satisfaction, reasoning correctness, visual decidability/reference fidelity, critical failure). The authors emphasize that official benchmark labels, rationales, and checklist annotations are never exposed to the self-evolution loop; benchmark evaluators run only for offline reporting. This separation matters because it rules out label leakage as an explanation for cross-round gains—an assumption that nonetheless depends on the memory judge being well-calibrated, which the paper asserts rather than independently validates.
Across rounds, positive records are retrieved as candidate strategies and negative records as warnings or repair constraints, yielding test-time self-evolution through external state updates rather than parameter updates.
Experimental results
Evaluation uses WISE (world-knowledge categories: cultural, temporal, spatial, biology, physics, chemistry) and Mind-Bench (knowledge-driven and reasoning-driven subcategories), reporting second-round results on 8×A100 GPUs with GPT-5.5 as the VLM controller.
| Model |
WISE Overall |
Mind-Bench Overall |
| GPT-Image-1 |
0.80 |
0.17 |
| Nano Banana Pro |
0.87 |
0.41 |
| Gen-Searcher-8B + Qwen-Image |
0.77 |
— |
| Mind-Brush |
0.78 |
0.31 |
| Qwen-Image (backbone) |
0.62 |
0.02 |
| MemoGen |
0.91 |
0.52 |
The headline claim is strong: after only two evolution rounds, MemoGen built on the open-source Qwen-Image backbone surpasses proprietary systems Nano Banana Pro and GPT-Image-1 on both benchmarks. On WISE it leads most knowledge-intensive categories (Cultural 0.96, Time 0.90, Chem 0.85); on Mind-Bench it exceeds Nano Banana Pro by 0.11 overall and Mind-Brush by 0.21. The Mind-Brush comparison is the most informative ablation-by-proxy: both systems use external knowledge and agentic reasoning, so the gap isolates the contribution of cross-task experience reuse relative to single-round retrieval and refinement.
Qualitative examples reinforce the mechanism: MemoGen renders a physically consistent unlit candle in space by applying negative constraints learned from prior failures of the candle–flame association, correctly depicts dissolved potassium permanganate as a homogeneous purple solution, and grounds celebrity-plus-product compositions that baselines fail.
Evolution analysis attributes gains in two stages: base-to-Round 1 reflects the agentic architecture itself (decomposition, retrieval, inspection) even without memory, while Round 1-to-Round 2 reflects experience-driven evolution. Round 3 saturates at roughly Round 2 performance, which the authors attribute candidly to two factors—the intrinsic ceiling of the frozen generator and noise or misjudgments from the automatic evaluator. This concession is significant: part of the reported headroom between MemoGen and its baselines may be bounded by evaluator reliability rather than genuine generation quality.
Limitations and open questions
Several limitations bear directly on interpretation. First, the framework's gains depend on the quality and calibration of the independent memory judge; systematic judge errors would be written into memory as polarity-labeled lessons and could propagate across rounds, and the paper does not quantify judge accuracy against ground truth. Second, saturation at Round 3 indicates that experience memory cannot exceed the frozen generator's synthesis capability—cases requiring rendering fidelity beyond the backbone remain unfixable at the agent layer. Third, evaluation reruns the same task set across rounds, so measured improvement partly reflects memorization of task-specific solutions; generalization to genuinely novel but similar tasks is asserted by the relation-level abstraction of memory records but not directly measured. Fourth, the reported comparison relies on automatic checklist-based evaluation, whose noise the authors themselves acknowledge. Finally, the compute cost of multi-round autonomous episodes (multiple generations, inspections, and searches per sample) is not analyzed, leaving open whether the accuracy gains justify inference-time overhead relative to fine-tuning approaches such as GenEvolve.
Conclusion
MemoGen demonstrates that explicit, polarity-labeled experience memory can function as a continual learning signal for T2I agents without any parameter updates, enabling an open-source backbone to exceed leading proprietary systems on knowledge- and reasoning-intensive benchmarks. The core empirical result—0.91 on WISE and 0.52 on Mind-Bench after two rounds—is substantial, though its interpretation rests on the calibration of the internal memory judge and on within-distribution task reuse. The work establishes cross-round, cross-task experience reuse as a dimension absent from prior single-episode agentic generation methods, and leaves open how well accumulated experience transfers beyond the evaluated task distributions.
