PromptSculptor: Multi-Agent Prompt Refinement

• PromptSculptor is a multi-agent framework that converts vague and abstract prompts into explicit, detailed instructions for image generation.
• It leverages specialized agents for intent inference, scene enrichment, self-evaluation, and feedback tuning, reducing user modification rounds from 6.08 to 2.35.
• The system’s model-agnostic design integrates with backends like Stable Diffusion XL and Midjourney, achieving top scores in PickScore and Aesthetic evaluations.

Searching arXiv for the directly relevant paper and closely related “Sculptor” methods to ground the article in current literature. PromptSculptor is a multi-agent prompt optimization framework for text-to-image generation that converts a short, vague, or abstract user prompt into a detailed, context-aware prompt intended to produce higher-quality images with fewer rounds of revision. Rather than treating prompt engineering as a single-step text expansion problem, it decomposes optimization into specialized agents for intent inference, scene and style enrichment, self-evaluation, and feedback-driven refinement. The framework is described as model-agnostic because it operates at the prompt layer rather than by fine-tuning the image generator, and it is reported with backends such as Stable Diffusion XL and Midjourney (Xiang et al., 15 Sep 2025).

1. Conceptual scope and problem formulation

PromptSculptor addresses automatic prompt optimization for cases in which the initial user input is short, underspecified, vague, metaphorical, abstract, or missing visual detail. The motivating claim is that strong text-to-image outputs usually require prompts that specify scene composition, style, background, lighting, color, mood, technical wording, and visually concrete descriptions, whereas many users provide only a simple idea or theme. The framework is explicitly designed to infer latent intent, concretize abstraction, enrich scene context, evaluate alignment, and then incorporate user feedback iteratively (Xiang et al., 15 Sep 2025).

A canonical example in the paper is: “draw a painting as a birthday blessing for my friend, he is like a lion.” The point of the example is that a literal text-to-image pipeline may generate an actual lion, whereas the intended semantics may concern strength, courage, majesty, or confidence. PromptSculptor therefore treats prompt optimization as a semantic interpretation problem as much as a lexical expansion problem. This distinguishes it from simple prompt extension methods that add generic detail without resolving metaphor or hidden context (Xiang et al., 15 Sep 2025).

The framework is presented as a response to several shortcomings in prior and manual workflows. Manual prompt engineering is described as labor-intensive and iterative; earlier automatic systems are said to focus too much on parameter editing while failing to enrich background, scene detail, and contextual depth; and many methods are described as lacking a feedback loop after image generation. PromptSculptor is accordingly structured as an iterative system in which prompt revision can continue after automated evaluation and after direct user comments (Xiang et al., 15 Sep 2025).

2. Agent architecture and internal decomposition

PromptSculptor is organized around four named agents. The architecture is motivated by the claim that different subtasks require different forms of reasoning: semantic interpretation, visual concretization, multimodal alignment checking, and interactive tuning. The paper argues that this multi-agent system improves language understanding over a traditional single-agent system by making these responsibilities explicit (Xiang et al., 15 Sep 2025).

Agent	Primary role	Main inputs
Intent Inference Agent	Infer explicit and latent intent	Original prompt $P_o$
Scene and Style Agent	Build a vivid visual prompt	Inferred intent and hidden context
Self-Evaluation Agent	Check image–prompt alignment	Generated image $I$, $P_o$, optimized prompt $P_{opt}$, threshold $\tau$
Feedback and Tuning Agent	Incorporate user comments	User feedback and current prompt

The Intent Inference Agent performs deep semantic analysis of the original prompt. It is intended to recover explicit requests, latent meaning, missing context, emotional tone, and metaphorical content. The paper states that this agent uses Chain-of-Thought reasoning to document step-by-step interpretation, especially when prompts contain nonliteral or symbolic language. In the framework’s logic, this stage extracts the “core idea and missing details” before any scene rendering language is added (Xiang et al., 15 Sep 2025).

The Scene and Style Agent takes the inferred semantics and transforms them into a visually actionable prompt. Its enrichment space includes subject depiction, medium, environment, lighting, color, mood, and composition. The paper presents this stage as analogous to an artist choosing concrete objects, settings, and stylistic devices to visualize an abstract concept. The role of this agent is therefore not merely lexical elaboration, but semantic concretization: it must decide which scene elements best externalize the inferred intent (Xiang et al., 15 Sep 2025).

The Self-Evaluation Agent acts as the system’s automatic alignment-checking component. Unlike the previous two agents, which operate mainly in language space, this agent compares generated images against prompt semantics using vision-LLMs. It is the only part of the method with explicit pseudocode and threshold-based control flow in the paper (Xiang et al., 15 Sep 2025).

The Feedback and Tuning Agent applies user comments after image generation. This stage is important because the framework acknowledges that alignment to the original prompt is not identical to alignment to the user’s actual preference. The original wording may be incomplete, and even an automatically improved prompt may fail to capture what the user realizes they want after inspecting outputs. The feedback agent therefore refines the prompt further until the image meets user expectations (Xiang et al., 15 Sep 2025).

3. Optimization loop and mathematical formulation

The operational pipeline begins with the user’s original prompt, passes through the Intent Inference Agent and the Scene and Style Agent to produce an optimized prompt, generates an image with a text-to-image backend, and then invokes the Self-Evaluation Agent to determine whether further refinement is required. If the user supplies additional comments, the Feedback and Tuning Agent continues the loop (Xiang et al., 15 Sep 2025).

The most explicit formal component is the Self-Evaluation Agent. It first computes a CLIP similarity score between the generated image and the original prompt:

$s \gets \text{CLIP}(I, P_o)$

where $I$ is the generated image and $P_o$ is the original prompt. If the similarity exceeds a threshold $\tau$, the current optimized prompt is retained. If not, the generated image is captioned with BLIP-2:

$C \gets \text{BLIP2}(I)$

and the prompt is revised by comparing the original prompt, the current optimized prompt, and the image caption:

$I$0

The paper states the stopping condition as $I$1, but does not specify the numerical value of $I$2. It also does not provide an explicit formula for how discrepancies between $I$3, $I$4, and $I$5 are scored internally during revision (Xiang et al., 15 Sep 2025).

The broader workflow can therefore be understood as a prompt-level iterative control loop rather than a learned end-to-end optimizer. There is no global training loss, no reinforcement-learning reward, and no parameter update to the image generator. The method instead relies on LLM-based reasoning and VLM-based alignment checking at inference time. This suggests that PromptSculptor functions primarily as a prompt middleware layer between the user and a downstream text-to-image backend (Xiang et al., 15 Sep 2025).

Chain-of-Thought is central to the method’s language-side behavior but is not formalized mathematically. The paper states that CoT is integrated into intent understanding and scene enrichment, improving interpretability by exposing step-by-step rationales. No CoT-specific ablation is reported, so its isolated quantitative contribution is not established in the paper (Xiang et al., 15 Sep 2025).

4. Implementation and empirical results

The multi-agent system is implemented with the OpenAI API and uses GPT-4o as the base model for agent reasoning. The Self-Evaluation Agent uses CLIP for text-image similarity and BLIP-2 for image caption generation. The framework is tested with Stable Diffusion XL and Midjourney as image-generation backends. Although the paper includes figures titled “Prompt display” for the different agents, the actual prompt templates are not reproduced as machine-readable text in the accessible material, and several practical settings are left unspecified, including temperature, token limits, iteration caps, prompt length caps, and random seeds (Xiang et al., 15 Sep 2025).

The evaluation uses a human-created set of 60 real-world prompts produced by 20 volunteers. These prompts were intentionally vague and included abstract words and metaphors. Automatic evaluation is performed on the generated images rather than directly on prompt text. The reported metrics are CLIP Score, PickScore, and Aesthetic Score. The paper describes these, respectively, as measuring prompt–image similarity, prompt–image alignment plus overall quality, and human preference on image aesthetics (Xiang et al., 15 Sep 2025).

Method	CLIP Score $I$6	PickScore $I$7	Aesthetic Score $I$8
Original	0.289	19.43	5.87
Extended	0.232	20.28	6.21
MagicPrompt	0.246	18.69	6.11
Ours w/o SEA	0.257	20.26	6.68
Ours	0.263	21.31	6.96

These results position PromptSculptor as best on PickScore and Aesthetic Score, while the original prompt obtains the highest CLIP Score. The paper explicitly explains this CLIP behavior as a metric artifact: because CLIP is computed between the original prompt and the generated image, images generated from the original prompt can score highly by remaining closer to that exact wording, even if they are less visually rich or less satisfying overall (Xiang et al., 15 Sep 2025).

Human evaluation reports both a preference score and the number of prompt-modification runs needed to reach satisfaction:

Method	Preference Score $I$9	Number of Runs $P_o$0
Original	69.85%	6.08
Extended	75.32%	4.22
MagicPrompt	67.28%	5.33
Ours	80.12%	2.35

The reduction from 6.08 runs for the original prompt to 2.35 for PromptSculptor is one of the method’s strongest practical claims. The paper also includes a limited ablation on the Self-Evaluation Agent, showing improvements from “Ours w/o SEA” to “Ours” across all three automatic metrics. No ablations are reported for Chain-of-Thought, the feedback agent, the number of agents, or the agent ordering. The experimental section also mentions PromptAgent as a baseline in setup, but PromptAgent does not appear in the results tables, which is an internal inconsistency in the report (Xiang et al., 15 Sep 2025).

5. Relation to adjacent “Sculptor” and prompt-optimization research

PromptSculptor belongs to a broader family of systems that treat prompting as an iterative, structured, or modality-specific authoring process rather than as one-shot text entry. A close conceptual neighbor is SCULPT, which optimizes long prompts for black-box LLMs by representing prompts as hierarchical trees and refining them through a Critic-Actor loop with beam search and UCB-based candidate selection. SCULPT is not a text-to-image system and is focused on long, structured prompts rather than short creative prompts, but it shares the idea that prompt optimization should be decomposed into interpretable subproblems rather than handled as flat rewriting (Kumar et al., 2024).

Promptify is another neighboring system, but from a systems/HCI perspective. It supports subject ideation, style extension, image layout and clustering, and prompt refinement suggestions for text-to-image generation. Compared with PromptSculptor, Promptify emphasizes interactive prompt exploration and image-browsing support rather than a multi-agent automatic optimization loop. Its architecture uses GPT-3.5-Turbo, Stable Diffusion v1.5, CLIP embeddings, t-SNE, affinity propagation, and CLIP Interrogator, making it a workflow environment for iterative prompt refinement rather than a four-agent reasoning framework (Brade et al., 2023).

Several other “Sculptor” methods in the literature are conceptually adjacent but belong to different problem domains. “Flux-Sculptor” addresses text-driven portrait editing through Prompt-Aligned Spatial Locator and Structure-to-Detail Edit Control on a FLUX.1-dev rectified-flow backbone, focusing on localization and identity-preserving image editing rather than prompt optimization (He et al., 5 Jul 2025). “Text2VDM” targets text-to-vector displacement maps for reusable sculpting brushes in 3D modeling software, using score distillation sampling over deformed planar meshes and addressing semantic coupling in local geometry synthesis (Meng et al., 27 Feb 2025). These systems share the general motif of “sculpting” as controlled iterative generation, but they operate on different objects: portrait edits, 3D brush assets, or optimized prompts.

This comparison situates PromptSculptor as a prompt-layer orchestration method rather than a generator or editor in its own right. Its distinctive contribution within this cluster is the use of a multi-agent prompt reasoning pipeline combined with automatic vision-language self-evaluation and explicit user-feedback tuning (Xiang et al., 15 Sep 2025).

6. Limitations, interpretation, and significance

PromptSculptor’s reported strengths are clear: it explicitly targets vague and abstract prompts, resolves metaphor and hidden intent through an Intent Inference Agent, enriches visual specificity through a Scene and Style Agent, reduces user effort through a Self-Evaluation Agent, and preserves interactivity through a Feedback and Tuning Agent. Its prompt-level design also makes it portable across multiple text-to-image backends without retraining the image generator (Xiang et al., 15 Sep 2025).

The paper also leaves several important details unspecified. The exact value of the Self-Evaluation threshold $P_o$1 is absent. The agent prompt templates are not available in machine-readable form. There is no reported iteration budget, no threshold-sensitivity analysis, no backend-specific split between SDXL and Midjourney, and no detailed study of how user feedback is represented internally. The evaluation uses 60 prompts from 20 volunteers rather than a public benchmark, which limits external comparability. The ablation evidence is confined to the Self-Evaluation Agent, so the individual empirical effect of the multi-agent decomposition, CoT, and feedback tuning is not isolated (Xiang et al., 15 Sep 2025).

The appendix-style discussion also shows that some abstract themes remain difficult. For “Love Transcends Distance,” the paper explicitly states that none of the prompts, including its own, clearly emphasized distance strongly enough. This is an important corrective to the broader claims of metaphor handling: the framework appears effective on many abstract prompts, but not uniformly so (Xiang et al., 15 Sep 2025).

Within the prompt-optimization literature, PromptSculptor is best understood as a specialized text-to-image prompt orchestration framework. It does not optimize prompts through a formal objective over model parameters, nor does it learn a prompt editor end to end. Instead, it combines LLM-based semantic decomposition with VLM-based post-generation checking and a human-in-the-loop correction stage. In encyclopedic terms, its significance lies in reframing prompt engineering as a staged reasoning-and-alignment process: infer what is meant, render it into visual language, verify what was generated, and revise until the result is satisfactory (Xiang et al., 15 Sep 2025).