Crafter: A Multi-Agent Harness for Editable Scientific Figure Generation from Diverse Inputs

Abstract: Scientific figures are among the most effective means of communicating complex research ideas, yet producing publication-quality illustrations remains one of the most labor-intensive parts of paper preparation. Existing automated systems each target a single figure type under text-only input, leaving the diversity of types and conditions researchers actually use unaddressed; their raster outputs further cannot be locally revised. Because scientific figures are structured compositions of discrete semantic components, the localized errors generators produce on such layouts demand not a stronger backbone but a harness. We instantiate this harness in two complementary systems: Crafter, a multi-agent harness for figure generation that generalizes across figure types and input conditions without architectural changes, and CraftEditor, which applies the same pattern to convert raster outputs into editable SVGs. Moreover, we introduce CraftBench, a benchmark spanning three figure types and four input conditions with human quality annotation. Experiments show that Crafter substantially outperforms both standalone generators and the agentic baseline on PaperBanana-Bench and CraftBench, with ablations confirming each component's independent contribution; CraftEditor faithfully converts outputs into editable SVGs that surpass all baselines. Our code and benchmark are available at https://github.com/HaozheZhao/Crafter.

• The paper presents a structured multi-agent harness called Crafter to generate editable scientific figures from diverse inputs.
• It combines iterative plan generation, structured critique, and specification refinement to achieve robust performance improvements.
• The approach also introduces CraftEditor for converting raster figures to editable SVGs, ensuring semantic accuracy and layout fidelity.

Harnessing Multi-Agent Orchestration for Editable Scientific Figure Generation

Overview

"Crafter: A Multi-Agent Harness for Editable Scientific Figure Generation from Diverse Inputs" (2605.30611) introduces a unified, harness-based architecture for the systematic generation and subsequent editing of scientific figures across diverse types and conditioning regimes. The paper fills crucial gaps in prior work, specifically limited generalization across figure modalities, input conditions, and the typical absence of structural editability in outputs. This is achieved by combining two core systems—Crafter, a multi-agent orchestrator for generation, and CraftEditor, a similarly harnessed system for editable raster-to-SVG conversion—underpinned by structured, iterative correction and verification strategies operating over an evolving specification.

Motivation and Problem Formulation

Automated scientific figure creation remains essential but underdeveloped relative to advances in text-to-image synthesis. The scientific illustration domain presents three distinctive challenges:

1. Combinatorial diversity: Figures span diagrams, posters, infographics, and nearly always require iterative, multimodal conditioning.
2. Structural precision: Layouts are complex, semantically dense, and require high reliability in the rendering and placement of discrete components (e.g., labeled modules, arrows, icons).
3. Editability: Real-world workflows necessitate local, element-wise revision, which raster-only outputs and most current pipelines cannot support.

Prior approaches have either focused narrowly on text-to-raster pipelines (e.g., [zhu2026paperbanana], [zhu2026autofigure]) or code-based generation (e.g., TikZ), each failing to generalize, integrate multi-modal inputs, or preserve downstream editability. Moreover, output images are static, and subsequent refinement via prompt or code revisions introduces inconsistency and prompt bloat, leading to semantic drift and sub-optimal results.

Crafter: Multi-Agent Harness Architecture

The Crafter harness establishes a structured, agentic orchestration loop around any black-box figure executor (e.g., advanced diffusion models). The system is built upon five loosely coupled agents sharing a structured, evolving memory specification. The pipeline comprises:

• Intent Reasoner: Identifies communicative goals and seeds the shared specification $\mathcal{S}$, abstracting the user's high-level objective from the provided context and conditioning documents.
• Plan Generator $\mathcal{D}$: Proposes $K$ intent-conditioned candidate layout plans to maximize initial diversity over semantically plausible renderings.
• Image Generator $\mathcal{E}$: Renders all $K$ plans using an underlying image synthesis backend.
• Critic $\mathcal{V}$: Evaluates each candidate via directive diagnostics comprising multi-dimensional axes (content accuracy, layout coherence, labeling, etc.) instead of scalar ratings.
• Specification Refiner $\mathcal{R}$: Integrates the critic’s structured edits (typed operations) into the master specification, maintaining prompt coherence and preventing contradictory feedback found in free-text revision paradigms.
• Convergence Judge: Mediates loop termination, early accepts satisfying candidates, or reverts to the best prior output to mitigate non-monotonic iterative refinement behavior.

  Figure 1: Crafter architecture as a multi-agent loop for iterative plan generation, plan execution, structured critique, and spec-driven revision, over a persistent shared specification.

Diversity-Driven Plan Exploration

Rather than rely on single-shot sampling—which often traps the process in compositional local minima—the system strategically branches at the plan level, invoking the generator on parallel candidate framings. This not only leverages the variance inherent to advanced diffusion models but allows escape from critical initial planning mistakes (e.g., faulty grid vs. columnar splitting, misassigned semantic zones), with $K$ dynamically selected per-instance.

Structured Corrective Layer and Verify-then-Refine Loop

A key innovation is the exclusive use of typed, structured specification edits based on the critic’s diagnostics, as opposed to unstructured, accumulating prompt addenda. The approach enforces internal spec consistency, robustly localizes and corrects artifacts (e.g., garbled or missing labels, misroutes), and supports closed-loop, explainable iterative refinement for up to $T$ rounds or convergence.

CraftEditor: Raster-to-Editable SVG Conversion

For genuine post-hoc editability, raster outputs must be converted into vector (SVG) representations. CraftEditor instantiates the same harness abstraction in a three-phase pipeline:

1. Extraction: Replaces segmentation with instruction-driven cleaning to isolate semantically salient elements, guided by a vision-language agent.
2. Processing: Each asset is captioned, grounded, and classified as vector/raster, followed by per-element cropping.
3. Composition: Candidate SVG skeletons are generated, assets are injected, and a hybrid critic (VLM plus programmatic checks) drives iterative, multi-round correction (e.g., arrow alignment, text bounding, element overlap), supporting robust and editable SVG assembly.

   Figure 2: CraftEditor: sequential extraction, classification, and composition harnesses for converting raster figures to coordinate-faithful, editable SVGs.

Benchmarking with CraftBench

To critically quantify cross-type and cross-condition generalization, the paper introduces CraftBench. This 279-sample benchmark spans three figure types and four input conditions: text-to-image, mask-completion, key-element composition, and sketch-conditioned generation. Sourced from arXiv, conference posters, and research blogs, each sample is manually curated and passes a multi-stage quality pipeline with rigorous grad-level annotation.

Figure 3: Representative tasks from CraftBench, illustrating diversity in both input conditioning and figure type.

Experimental Evaluation

Crafter is evaluated against both vanilla generators (GLM-Image, Qwen-Image, GPT-Image-2, Nano Banana 2/Pro) and prominent agentic frameworks (PaperBanana, AutoFigure) on PaperBanana-Bench and CraftBench, utilizing referenced VLM-as-judge protocols. Key results:

• Crafter achieves state-of-the-art performance, improving over its image-generation backbone by $27.6$–$\mathcal{D}$0 points across benchmarks and tasks (e.g., $\mathcal{D}$1 vs. $\mathcal{D}$2 with Nano Banana 2 on PaperBanana-Bench).
• Generalization: Crafter outperforms all baselines across all figure types and input conditions, not just in a narrow regime.
• Ablation studies confirm the necessity of each harness mechanism, with the elimination of plan diversity or the corrective layer leading to degradations of $\mathcal{D}$3–$\mathcal{D}$4 points.
• Pluggability: Upgrading the underlying executor yields incremental rather than disruptive changes, confirming the harness’s independence from model details.

  Figure 4: Qualitative comparison highlighting Crafter’s consistency across input conditions, preserving structural fidelity and typographic detail.

For the editable-output component, CraftEditor is benchmarked against Edit-Banana and AutoFigure-Edit. It delivers superior seven-axis VLM-assessed output quality, especially for structural and typographic accuracy:

• CraftEditor overall score: $\mathcal{D}$5, compared to $\mathcal{D}$6 (AutoFigure-Edit) and $\mathcal{D}$7 (Edit-Banana), with ablations indicating that iterative harness-driven composition is crucial ($\mathcal{D}$8 drop without it).

  Figure 6: Editable-output system comparison; CraftEditor maintains semantic structure and editability, outperforming prior methods on text, arrow, and layout fidelity.

Failure Analysis

Despite significant improvements, residual failures remain, primarily:

• Intent Reasoner errors: Multi-panel captions may be collapsed to a single layout if not correctly parsed, propagating a ‘dropped panels’ error.
• Boundary discontinuities in mask completion: The infill may visually clash with preserved regions at the mask boundary.
• Under-determined sketch-conditioned outputs: The generator may reproduce the structural skeleton without populating essential illustrative content.

  Figure 8: Representative failure cases—missed panels, boundary artifacts, and undercompleted skeletons—isolating key remaining bottlenecks in the harness.

Implications and Future Directions

This work demonstrates that systematic, agentic orchestration and structured spec-driven correction can fundamentally strengthen robustness and generalizability in complex scientific image synthesis and editing tasks. Harness architectures efficiently decouple task planning, plan diversity, formal critique, and semantic correction from the idiosyncrasies of underlying generators, thus future-proofing pipelines as executors improve. The same methodology is extensible to structured-output domains, offering a practical path for scalable, verifiable, and editable content synthesis across scientific, educational, and technical documentation. Key research avenues include further coverage expansion in benchmarks (e.g., more infographics), refining intent reasoning and critic modules, and integrating more advanced execution and verification primitives.

Conclusion

Crafter and CraftEditor, leveraging multi-agent harnesses around black-box generators, set new standards for scientific figure synthesis and structural editability under diverse, realistic input conditions (2605.30611). The strict, compositional separation of planning, execution, and error correction yields robust performance gains over previous agentic and generative approaches and establishes a blueprint for next-generation structured visual content pipelines.