DataEvolver: Self-Evolving Multi-Agent Data Construction for Text-Rich Image Generation

Abstract: Text-rich image generation is one of the most challenging settings in image generation, since models must simultaneously produce visually realistic images and render legible, semantically aligned, and layout-consistent text. Existing data pipelines usually follow a static crawl-filter-freeze paradigm. They collect candidate samples, filter them once, and freeze the accepted data for training. However, rejected samples are usually discarded, although they often contain useful failure signals such as OCR errors and semantic mismatches. As a result, later construction rounds may repeat the same failure modes. To address these limitations, we propose DataEvolver, a self-evolving multi-agent framework for text-rich image data construction. DataEvolver treats data construction as feedback-driven construction policy evolution. A Retriever collects candidate samples, a Verifier assigns quality scores and rejection causes, a Critic summarizes round-level feedback into semantic feedback, and a Generator completes under-covered regions through targeted synthesis. The updated feedback memory then guides the next construction round. Experiments on text-rich image generation benchmarks show that DataEvolver produces more useful training data than fixed-dataset baselines under matched data budgets. At the 0.75M scale on PixArt-alpha, DataEvolver improves OCR-F1 over the strongest baseline by 85.3 percent on TextScenesHQ and 35.3 percent on LongTextBench. The improvements are consistent across both evaluated benchmarks and also transfer to Show-o2, indicating that the benefit of DataEvolver is not tied to a single downstream generator. These results suggest that rejected samples can provide actionable feedback for improving text-rich image data construction.

• The paper demonstrates an iterative, feedback-driven multi-agent framework that dynamically refines data construction policies to mitigate recurring errors in text-rich image generation.
• It employs four collaborative agents—Retriever, Verifier, Critic, and Generator—to harness semantic feedback and improve OCR accuracy, layout consistency, and overall dataset quality.
• Quantitative results show substantial gains in OCR-F1 scores over static baselines, validating the framework's effectiveness and scalability for complex data construction tasks.

DataEvolver: Feedback-Driven Multi-Agent Data Construction for Text-Rich Image Generation

Motivation and Problem Formulation

Text-rich image generation presents substantial challenges due to the requirement of simultaneously achieving visual realism, text legibility, semantic alignment, and layout consistency. Traditional data construction pipelines in this domain typically operate under a static crawl–filter–freeze approach, discarding samples that fail verification, despite these rejected samples encapsulating valuable information regarding failure modalities (e.g., OCR errors, semantic mismatches). This constraint leads to the repetition of the same failure patterns across construction rounds.

The "DataEvolver: Self-Evolving Multi-Agent Data Construction for Text-Rich Image Generation" (2606.31537) paper reconceptualizes multimodal data construction as an iterative policy refinement process, in which failure signals collected during dataset assembly are harnessed to dynamically update retrieval/generation strategies and mitigate recurring errors.

Figure 1: DataEvolver incorporates rejected sample feedback into a closed-loop pipeline for data policy improvement, leading to continual enhancements in OCR-F1 across scales compared to static baselines.

The DataEvolver Multi-Agent Framework

DataEvolver comprises four collaborative agents: Retriever, Verifier, Critic, and Generator, operating in a closed feedback loop.

• The Retriever adaptively generates semantically-driven queries to retrieve candidate samples based on evolving experience and prior feedback.
• The Verifier enforces a multidimensional quality control, including OCR-based text extraction, near-duplicate filtering, perceptual and semantic assessment, and comprehensive attribution of rejection causes.
• The Critic summarizes round-level feedback, transforming both aggregate statistics and rejection distributions into actionable, natural-language semantic feedback. This feedback is essential for query diversification, prompt refinement, and memory updates.
• The Generator targets coverage gaps by synthesizing samples for underrepresented topic–subtopic pairs, using prompts refined according to Critic-derived feedback.

  Figure 2: Schematic of the DataEvolver multi-agent data construction loop; rejected samples are leveraged as scaffolding for semantic policy updates and targeted data synthesis.

Through iterative rounds, DataEvolver jointly optimizes retrieval, prompt engineering, and synthetic data generation—moving beyond the constraints of purely static pipelines.

Critic-Guided Semantic Policy Updates

A central design in DataEvolver is the conversion of low-level rejection patterns into high-level, actionable policy refinements. The Critic agent, powered by LLMs, generates concise semantic feedback, which may include recommendations for query removal, template diversification, or specific prompt modifications addressing, for example, high duplicate rates or persistent OCR failures.

Figure 3: Critic-generated semantic feedback leads to measurable reductions in failure modes (e.g., deduplication errors) in subsequent construction rounds.

This closed-loop interaction ensures that the dataset construction process is not merely pass/fail filtering but is effectively learning from its own mistakes at a policy level.

Quantitative and Qualitative Results

DataEvolver is empirically validated on text-rich image generation benchmarks (TextScenesHQ and LongTextBench) with both retrieval- and synthesis-based baselines (MARIO-10M, AnyWord-3M). When fine-tuning state-of-the-art downstream generators (PixArt-$\alpha$ and Show-o2) under matched data budgets, DataEvolver achieves substantial gains in OCR-F1 score—improving over the strongest static baseline by 85.3% on TextScenesHQ and 35.3% on LongTextBench at the 0.75M scale. These improvements are sustained across both evaluated generation models, indicating generality across architectures.

Qualitative analyses reveal that DataEvolver-generated datasets alleviate critical failure cases such as off-topic generations, missing or distorted text regions, and chronic layout issues.

Figure 4: DataEvolver-generated training data drives more robust downstream generations, reducing frequent text-related artifacts seen in static baselines.

Furthermore, ablation studies demonstrate the necessity of both Critic (feedback) and Generator (targeted completion) modules:

• Removing the Critic sharply degrades OCR-F1, indicating that semantic feedback is crucial for suppressing recurring errors.
• Removing the Generator reduces tail coverage and overall diversity, leading to inferior performance on semantically diverse benchmarks.

Analysis of Failure Modes and Feedback

DataEvolver provides a detailed lens into the evolving distribution of rejection causes throughout the data construction pipeline, surfacing non-trivial issues such as:

Figure 5: Unreadable overlaid text—failure due to extreme distortion defeating OCR.

Figure 6: Tiny screen text—failure due to insufficient textual area for reliable supervision.

Figure 7: Perspective distortion leading to occlusion and background interference.

Figure 8: Sparse handwriting introducing verification inconsistency.

Figure 9: Cluttered scene text with low-density useful text regions.

By systematically categorizing these failure modalities and incorporating their patterns in construction-time feedback, DataEvolver moves towards adaptive mitigation rather than static rejection.

Implications and Future Directions

Practically, DataEvolver demonstrates that treating data construction as a dynamic, feedback-driven process enhances both the quality and coverage of datasets for complex generation tasks, particularly where semantic and structural nuance is critical (e.g., text-image alignment, OCR reliability). Empirical evidence supports the claim that dataset improvements arise during the data construction phase itself, not solely as a function of downstream model fine-tuning.

Theoretically, this approach situates data construction within a policy optimization landscape—akin to RL–inspired self-improvement but operating at the level of pipeline strategy rather than model parameters. This opens avenues for more generally applying such feedback-driven construction not only in text-rich settings, but in heterogeneous multimodal or instruction-oriented tasks, where verification and failure types may be even more diverse.

Further research could extend these methods to domains with weak/noisy supervision, endowing frameworks with the ability to discover new data modalities, bootstrap structured data collection, and autonomously steer data policy in domains with shifting targets or objectives.

Conclusion

DataEvolver redefines text-rich image data construction as an adaptive, multi-agent feedback system, using rejected samples as informative signals for policy-level refinement. The architecture validates that failure-aware, closed-loop data construction advances both empirical and structural dataset quality relative to static baselines. The modular interface between agents, policy updates, and generator enables extensibility to a broader class of multimodal dataset construction tasks—pointing towards general-purpose, autonomous data-centric AI pipelines built on continuous self-improvement.