IMAGAgent: Multi-turn Image Editing Framework

Updated 5 July 2026

IMAGAgent is a multi-turn image editing framework that iteratively plans, executes, and reflects on sub-tasks to reduce error accumulation, semantic drift, and structural distortion.
The system integrates dynamic historical context with constraint-aware planning, tool-chain orchestration, and multi-expert reflection to ensure coherent multi-turn edits.
Experimental results on MTEditBench and MagicBrush demonstrate improved visual consistency, semantic fidelity, and corrective feedback compared to traditional editing methods.

Searching arXiv for the specified topic and closely related agentic image-generation/editing work. IMAGAgent is a multi-turn image editing agent framework that casts image editing as a closed-loop, constraint-aware process organized around a “plan–execute–reflect” mechanism. It is designed for settings in which an initial image is repeatedly edited through a sequence of user instructions, with each intermediate output serving as the input to the next turn. The framework addresses three failure modes identified in existing multi-turn editing systems—error accumulation, semantic drift, and structural distortion—by combining instruction decomposition, dynamic tool scheduling, and multi-expert reflective correction within a unified pipeline (Shen et al., 12 Feb 2026).

1. Problem setting and formal definition

IMAGAgent targets multi-turn image editing, where a system receives an initial image and then iteratively applies editing instructions over successive turns. In this regime, each intermediate result becomes the starting point for the next edit, so the system must satisfy the current instruction while preserving the semantics and structure established so far. The paper characterizes this as a long-horizon setting in which later edits depend on earlier ones and small errors can propagate across turns (Shen et al., 12 Feb 2026).

The framework is formalized as a mapping

$\mathcal{I}^* = \Phi(\mathcal{I}_0, \mathcal{T}_0),$

where $\mathcal{I}_0$ is the initial image, $\mathcal{T}_0$ is the user instruction, and $\mathcal{I}^*$ is the final edited image. To support iterative control, IMAGAgent maintains a dynamic historical context

$\mathcal{C} = \{ (\mathcal{R}_k, \mathcal{I}_k, \mathcal{F}_k) \}_{k \ge 1} \cup \{ \mathcal{I}_0 \},$

in which each completed editing attempt stores an orchestration plan $\mathcal{R}_k$ , an output image $\mathcal{I}_k$ , and feedback $\mathcal{F}_k$ (Shen et al., 12 Feb 2026).

The motivating diagnosis is explicit. Existing paradigms often treat each instruction in isolation, lack a persistent context representation, and do not evaluate whether an edit succeeded before moving forward. The consequence is a characteristic triad of failures: error accumulation, because early mistakes are not corrected; semantic drift, because prior constraints are not explicitly represented; and structural distortion, because local edits are not subjected to robust visual verification (Shen et al., 12 Feb 2026).

2. Closed-loop architecture

The system is organized into three main modules: Constraint-Aware Planning, Tool-Chain Orchestration, and Multi-Expert Collaborative Reflection. These modules form an iterative control loop rather than a one-shot editing pipeline (Shen et al., 12 Feb 2026).

Module	Function	Main output
Constraint-Aware Planning	Decomposes complex instructions into atomic sub-tasks	$\mathcal{A} = \{t_1, t_2, \dots, t_n\}$
Tool-Chain Orchestration	Builds and executes a toolchain for the current sub-task	Candidate edited image $\mathcal{I}_i^{(j+1)}$
Multi-Expert Collaborative Reflection	Critiques the candidate edit and decides accept/retry	$\mathcal{I}_0$ 0

The planner $\mathcal{I}_0$ 1 receives the current image and instruction and produces a task sequence

$\mathcal{I}_0$ 2

For each sub-task $\mathcal{I}_0$ 3, an agent LLM generates an execution plan

$\mathcal{I}_0$ 4

which is then executed as

$\mathcal{I}_0$ 5

The resulting candidate is scored by a committee of VLM critics, and the fused reflection output determines whether the system accepts the edit or retries the same sub-task (Shen et al., 12 Feb 2026).

The acceptance rule is thresholded. If the reflection score $\mathcal{I}_0$ 6 exceeds the success threshold $\mathcal{I}_0$ 7, the candidate is accepted and the system advances to the next sub-task. Otherwise it re-edits, up to an iteration limit $\mathcal{I}_0$ 8. In the reported experiments, $\mathcal{I}_0$ 9 and $\mathcal{T}_0$ 0 (Shen et al., 12 Feb 2026).

A common misconception is that IMAGAgent is a new end-to-end editor. It is not. The paper defines it as an orchestration framework built from off-the-shelf models and hand-designed prompts; no new model is trained end-to-end, and no gradient-based optimization or reinforcement learning is described (Shen et al., 12 Feb 2026).

3. Constraint-aware planning

The planning module is intended to convert a high-level natural-language instruction into a sequence of atomic edit instructions that are executable, visually grounded, and ordered by dependency. The planner is implemented with Qwen-VL-MAX, which takes the image and instruction and returns a JSON-like array of sub-task strings (Shen et al., 12 Feb 2026).

The planner is governed by three explicit constraints:

Target singularity: each sub-task operates on a single entity or coherent group.
Semantic atomicity: a sub-task cannot be meaningfully decomposed further without changing semantics or adding redundancy.
Visual perceptibility: each sub-task must produce a tangible visual change.

These constraints are supplemented by temporal dependency rules and clarity requirements. If one edit depends on an object introduced by another, the creation step must precede the dependent step. The output sequence is therefore not merely segmented but also reordered into a causally consistent editing plan (Shen et al., 12 Feb 2026).

The paper gives a representative decomposition example for the instruction:

$\mathcal{T}_0$ 8

This example illustrates the planner’s role in separating heterogeneous edit types—attribute modification, pose editing, environment replacement, object insertion, object removal, and object addition—into units that downstream detection, segmentation, and editing tools can address (Shen et al., 12 Feb 2026).

The ablation results indicate that this constraint regime is not incidental. Removing target singularity, semantic atomicity, and visual perceptibility lowers MTEditBench performance to DINO 0.722, CLIP-I 0.833, and CLIP-T 0.229, with the paper explicitly noting that instruction alignment degrades when decomposition is not well constrained (Shen et al., 12 Feb 2026).

4. Tool-chain orchestration

Once a sub-task is defined, IMAGAgent dynamically constructs an execution path conditioned on the current image, the current sub-task, and the historical context. The controller is GLM-4.1V-9B-Thinking, which reasons over the current state and emits a Python function ToolGenerate(img) implementing the selected toolchain (Shen et al., 12 Feb 2026).

The tool inventory is heterogeneous and includes image retrieval, detection, segmentation, inpainting, and editing:

Editing / inpainting: Inpainting, InpaintingByIpAdapter, ImageEditByPipe, ImageEditByAPI
Detection & segmentation: GroundingDINO_SAM2
Image retrieval: GetTargetImage
Underlying editing models: Doubao-Seedream-4.0, Qwen-Image-Edit, Stable-Diffusion-XL-Base-1.0

GroundingDINO_SAM2 returns target_box, maxscore, box_image, original_mask, white_mask, and cutOut_img; internally it uses Grounding DINO Base for detection and SAM3 for segmentation. GetTargetImage(target: str) → Image retrieves a reference image consistent with a textual description using CLIP-ViT-Large-Patch14 (Shen et al., 12 Feb 2026).

The orchestration policy is prompt-based rather than learned. The controller checks prior edits and prior failures through $\mathcal{T}_0$ 1, including negative prompts such as "background changed" or "dog", and can bias future tool selection accordingly. This history-sensitive behavior is central to preserving previously established image content (Shen et al., 12 Feb 2026).

The reported implementation stack includes Qwen-VL-MAX as planner, GLM-4.1V-9B-Thinking as orchestrator, SAM3 for segmentation, Grounding-DINO-Base for detection, CLIP-ViT-Large-Patch14 for retrieval, and Doubao-Seedream-4.0 as the main image editing API. The system runs on a single NVIDIA A100 GPU, with local hosting except for API-based proprietary models. The paper estimates a per-task cost of approximately 0.0174 USD under its reported API pricing and a $\mathcal{T}_0$ 2 resolution basis (Shen et al., 12 Feb 2026).

5. Reflection, feedback, and adaptive correction

After each candidate edit is generated, IMAGAgent performs multi-expert collaborative reflection. Three independent VLM experts critique the candidate using the pre-edit image, the edited image, and the current sub-task instruction, producing

$\mathcal{T}_0$ 3

where $\mathcal{T}_0$ 4 is positive feedback, $\mathcal{T}_0$ 5 is a negative prompt string, and $\mathcal{T}_0$ 6 is a score that combines semantic alignment, perceptual quality, aesthetic assessment, and logical consistency (Shen et al., 12 Feb 2026).

These critiques are fused by an aggregator LLM: $\mathcal{T}_0$ 7 The aggregator is DeepSeek-V3.2. The expert committee consists of Qwen-VL-MAX, Doubao-Seed-1.6-Vision, and Doubao-Seed-1.6. The fused negative prompts are passed directly into downstream editing tools, while positive prompts are used to preserve content that was judged correct (Shen et al., 12 Feb 2026).

The feedback mechanism is operational rather than merely diagnostic. In the paper’s “remove power sockets” example, the initial edit removes the sockets but distorts the wall and introduces an unintended bottle. Reflection then produces negative prompts such as "bottle" and "background changed" together with a positive instruction to remove the sockets while keeping the wall unchanged. On the second execution, the orchestrator selects a more precise detection-and-inpainting path, yielding a corrected result with preserved wall structure (Shen et al., 12 Feb 2026).

The reported refinement statistics quantify the importance of this loop. Approximately 68.32% of steps satisfy the quality threshold on the first attempt, and approximately 27% initially fail but are successfully corrected within three iterations. The authors note that without the loop those failures would remain and would contribute to severe multi-turn degradation. The multi-expert reflection stage increases runtime by approximately 1.4× relative to open-loop baselines (Shen et al., 12 Feb 2026).

The reflection module also resolves a second misconception: IMAGAgent is not only a planner with tool use. Its distinctive property is that planning, execution, and critique are coupled through a persistent memory of failed attempts and corrective prompts, so the system can revise its own behavior at the level of both tool selection and prompt design (Shen et al., 12 Feb 2026).

6. Benchmarks, results, and ablations

Because existing multi-turn editing benchmarks were considered shallow, the paper introduces MTEditBench, constructed from the VINCIE training corpus through heuristic instruction segmentation, random shuffling, VLM-based segmentation and validation, filtering for sequences with at least 4 turns, and manual verification. The final benchmark contains 1,000 long-horizon sequences, each with at least four turns, spanning object addition, removal, and attribute editing (Shen et al., 12 Feb 2026).

Evaluation uses three automatic metrics:

DINO for visual consistency / identity
CLIP-I for semantic similarity between original and edited images
CLIP-T for alignment between edited image and editing instruction

On MTEditBench, IMAGAgent reports average scores of DINO 0.766, CLIP-I 0.875, and CLIP-T 0.248. The paper emphasizes that degradation over turns is substantially slower than for comparison systems: for example, OmniGen drops in DINO from 0.800 at turn 1 to 0.480 at turn 5, whereas IMAGAgent drops from 0.803 to 0.721 over the same span (Shen et al., 12 Feb 2026).

On MagicBrush, which contains more than 10k triplets from 5,313 sessions and has a maximum of 3 turns, IMAGAgent reports DINO 0.806, CLIP-I 0.898, and CLIP-T 0.282, outperforming all non-proprietary baselines across metrics and turns. GPT‑4o attains stronger CLIP-T but is proprietary (Shen et al., 12 Feb 2026).

Ablation results further localize the gains. Removing reflection reduces MTEditBench performance to DINO 0.689, CLIP-I 0.815, and CLIP-T 0.221. Removing historical context reduces performance further to DINO 0.646, CLIP-I 0.772, and CLIP-T 0.233, with the paper specifically highlighting the large CLIP-I drop as evidence that historical context is critical for identity and semantic consistency. Replacing the multi-expert committee with a single expert yields DINO 0.742, CLIP-I 0.854, and CLIP-T 0.237, indicating a consistent advantage for collaborative reflection (Shen et al., 12 Feb 2026).

The user study covers 50 cases with 20 expert raters and reports that IMAGAgent is ranked top in instruction following, visual realism, and overall quality. The code is available at https://github.com/hackermmzz/IMAGAgent.git (Shen et al., 12 Feb 2026).

7. Position within agentic image systems and limitations

IMAGAgent belongs to a broader arXiv trend that treats image systems as agents rather than single-pass generators or editors. In adjacent work, ImAgent is a training-free unified multimodal agent for test-time scalable image generation and editing that integrates reasoning, generation, and self-evaluation inside a single backbone (Wang et al., 14 Nov 2025). Qwen-Image-Agent frames image generation around a “Context Gap” and organizes plan, reason, search, memory, and feedback as context construction before rendering (Zhang et al., 25 Jun 2026). By contrast, IMAGAgent is explicitly specialized for multi-turn image editing, and its core novelty lies in coupling constraint-aware task decomposition, heterogeneous tool scheduling, and multi-expert visual reflection within a persistent historical context (Shen et al., 12 Feb 2026).

Within the paper’s own positioning, IMAGAgent differs from several prior categories. Relative to single-turn diffusion editing systems such as SDEdit, DiffEdit, InstructPix2Pix, and Imagic, it adds multi-step planning over a single instruction together with explicit feedback and self-correction. Relative to unified multi-task image generators/editors such as OmniGen, ACE++, and VAREdit, it replaces an open-loop “instruction → edit” paradigm with dynamic toolchain orchestration and reflection-driven correction. Relative to in-context and autoregressive multi-turn editing systems such as VINCIE and ICEdit, it adds an explicit visual evaluation stage with a multi-expert VLM committee (Shen et al., 12 Feb 2026).

The framework’s limitations are also clear. It is currently restricted to static images; extension to video and 3D is presented as non-trivial because of temporal consistency and multi-view geometric coherence. Performance depends heavily on proprietary foundation models such as Qwen, GLM, Doubao, and DeepSeek. Inference cost and latency increase because of the closed-loop reflection stage. Finally, the control logic is heuristic / prompt-based rather than a learned policy, so there is no explicit guarantee of optimal decomposition or tool selection (Shen et al., 12 Feb 2026).

A plausible implication is that IMAGAgent is best understood as a systems architecture rather than a single editing model. Its contribution is procedural: it formalizes long-horizon image editing as a stateful sequence of planned sub-tasks, executable toolchains, and reflective corrections. In that sense, it establishes a concrete blueprint for agentic editing systems in which semantic preservation and structural stability are enforced through orchestration rather than assumed to emerge from a single forward pass (Shen et al., 12 Feb 2026).