GPT-IMAGE-EDIT-1.5M: Image Editing Benchmark

Updated 3 July 2026

GPT-IMAGE-EDIT-1.5M is a large-scale, instruction-driven corpus comprising over 1.5M triplets that link natural-language edits to source and edited images.
It unifies major open datasets with a rigorous three-stage pipeline, including output regeneration and automated instruction rewriting, to ensure high fidelity across multiple aspect ratios.
The dataset underpins state-of-the-art model development with benchmarks like GEdit-EN and ImgEdit-Full, narrowing performance gaps with proprietary systems.

GPT-IMAGE-EDIT-1.5M is a large-scale, instruction-driven image-editing corpus and research benchmark. It comprises over 1.5 million high-quality triplets—{instruction, source image, edited image}—systematically curated and refined to advance the development and evaluation of instruction-following image editing models. This dataset forms the backbone for state-of-the-art open-source models and unified multimodal architectures, narrowing performance gaps with proprietary systems.

1. Dataset Definition and Source Composition

GPT-IMAGE-EDIT-1.5M consists of more than 1.5 million data points, each a triplet containing: (i) a natural-language editing instruction, (ii) a source image, and (iii) a corresponding edited image strictly executing the instruction. Images are provided at fixed aspect ratios— $1{:}1$ (1024×1024), $3{:}2$ (1536×1024), and $2{:}3$ (1024×1536)—with approximate distribution of 40%, 30%, and 30% across the corpus (Wang et al., 28 Jul 2025). The dataset unifies and refines three major open image-editing datasets: OmniEdit, HQ-Edit, and UltraEdit, chosen for their variety of edit types but previously limited by either noisy instructions or suboptimal visual fidelity.

Edit operations are broadly distributed as follows: add/remove/replace subject (~25%), background change (~12%), color/texture adjustment (~12%), style transfer (~10%), text insertion (~5%), tone/lighting (~8%), composite/complex edits (≥2 atomic ops, ~15%), and other categories (e.g., motion, portrait tweaks) covering the remaining spectrum (Wang et al., 28 Jul 2025).

2. Data Construction and Quality Control Pipeline

The construction pipeline involves three principal stages: output image regeneration, instruction rewriting, and full input–output pair regeneration where necessary.

a) Output Regeneration: For each original {instruction, source image} pair, the system queries GPT-4o’s “gpt-image-1” API, generating higher-fidelity edited images. All outputs adhere to one of the specified resolutions and undergo artifact filtering—specifically, padding artifacts must not exceed 0.5% residual border, and any generation failures are culled automatically. Upscaling/downscaling are applied to harmonize image sizes from all source datasets (Wang et al., 28 Jul 2025).

b) Instruction Rewriting: To improve semantic alignment, 10% of instructions (OmniEdit subset) are rewritten by GPT-4o, conditioned on the new output and the source image, further increasing instruction adherence (Wang et al., 28 Jul 2025).

c) Full Pair Regeneration: For 50% of HQ-Edit, both the source and edited images are newly synthesized by GPT-4o: first generating a source image given the instruction, then reapplying the instruction to that generated source for the edit pair (Wang et al., 28 Jul 2025).

All procedures are automated, with no manual annotation beyond automated checks. Filtering leverages learned scoring models and thresholds like CLIPScore ≥ 0.35, area coverage for object/background edits, and artifact/fake detection via dedicated models (Ye et al., 26 May 2025).

3. Metadata Schema and Benchmarks

Each data sample includes structured metadata such as references to source and edited images, object masks, bounding boxes, resolution, natural-language caption, edit instruction, edit category/type, and automated quality scores. JSON schema is standardized:

{
  "id": "IMG00012345",
  "ref_image": "images/ref/IMG00012345.jpg",
  "edited_image": "images/edit/Remove/IMG00012345_edit.jpg",
  "mask": "masks/Remove/IMG00012345_mask.png",
  "resolution": [1600, 2560],
  "caption": "A tabby cat sitting on a wooden deck.",
  "instruction": "Remove the plastic bottle lying next to the cat.",
  "edit_type": "Remove",
  "bbox": [400,1200,600,1400],
  "post_score": 4.7,
  "fake_score": 0.05
}

(Ye et al., 26 May 2025)

Benchmarks built on GPT-IMAGE-EDIT-1.5M leverage several standardized suites: GEdit-EN (11 edit types, instruction adherence, perceptual quality), ImgEdit-Full (9 task families), and Complex-Edit (three atomic edits per instruction for 50% of OmniEdit). Metrics include MLLM-based scoring for instruction following and perceptual quality, CLIPScore, FID, LPIPS, and composite evaluation for complex-editing performance (Wang et al., 28 Jul 2025).

4. Model Development, Training Paradigms, and Downstream Performance

Fine-tuning on GPT-IMAGE-EDIT-1.5M enables substantial improvements over previous open-source data regimes. Notably, FLUX.1 Kontext dev (flow-matching backbone) with Qwen-VL-7B and T5 text encoders achieves highly competitive results: GEdit-EN score 7.24, ImgEdit-Full 3.80, Complex-Edit overall 8.78, closely matching proprietary GPT-4o scores (Wang et al., 28 Jul 2025).

The canonical architecture (as in UniGen-1.5 (Tian et al., 18 Nov 2025)) leverages a 7B parameter transformer backbone, frozen SigLIP2 continuous encoders and discrete MAGViTv2 tokenizers for vision input, and a three-way condition concatenation strategy—continuous vision, text instruction, and discrete vision. Training proceeds through pre-training on multi-million image–text pairs, supervised finetuning on synthetic generation and editing pairs, a dedicated instruction alignment phase, and RLHF via Group Relative Policy Optimization (GRPO) with shared reward models (Tian et al., 18 Nov 2025).

Table: Benchmark Comparison (ImgEdit-Full)

Model	Overall Score
Ours (GPT-IMAGE-EDIT-1.5M → FluxKontext)	3.80
GPT-4o	4.20
Best previous open-source	4.00

(Wang et al., 28 Jul 2025)

5. Unified RLHF and Reward Optimization

The UniGen-1.5 approach shares a unified RL objective for text-to-image generation and image editing, using a family of reward models: CLIP-ViT-H cosine, HPSv2 human-preference, UnifiedReward-7B, and an ORM (“Yes/No” outcome). The reward for candidate $\hat{H}$ conditioned on $T$ is:

$R(\hat{H}, T) = \frac{R_C + R_H + R_U + R_O}{4}$

Policy optimization employs the GRPO algorithm, calculating normalized groupwise advantages and penalizing deviation from a reference policy via KL divergence. Post-supervised finetuning, a lightweight “Edit Instruction Alignment” phase (≈17,000 triplets) further sharpens the model’s capacity to map instructions to edits, increasing the reward variation necessary for effective RL (Tian et al., 18 Nov 2025).

Ablation confirms that both unified RL across tasks and explicit instruction alignment substantively boost editing and generation scores, with compositional conditions ( $\mathcal{X}_C^U \rightarrow \mathcal{T}_C \rightarrow \mathcal{X}_C^G$ ) found optimal (Tian et al., 18 Nov 2025).

6. Corpus Structure, Complexity, Expansion, and Limits

The dataset includes ≈1.35M single-turn edits (basic operations, novel/complex categories) and 0.15M multi-turn tasks for content memory, understanding, and version backtracking. Each edit type and complexity level is precisely quota-managed to avoid bias, using mining from LAION-5B, video frame extraction (Open-Sora Plan), and synthetic prompt chaining for category coverage (Ye et al., 26 May 2025). Automated judging and QA using large vision-LLMs (e.g., GPT-4o) provides per-sample scoring, with a subset (up to 10%) manually spot-checked.

Known limitations include reliance on proprietary models (e.g., GPT-4o) for output and instruction generation, degradation in realism/identity for high edit-complexity (C3+ in Complex-Edit), and persistent challenges in categories such as text insertion (Wang et al., 28 Jul 2025). Proposed directions include extending to multimodal scenarios (video, 3D, multi-view), hybrid crowdsourcing pipelines for annotation, and more sophisticated misalignment detectors.

7. Impact and Future Directions

GPT-IMAGE-EDIT-1.5M demonstrably closes the gap between open-source and proprietary instruction-based image editing. The corpus’s scale, compositional diversity, operational rigor, and benchmarked performance underpin its adoption for both dataset building and model development (Wang et al., 28 Jul 2025, Tian et al., 18 Nov 2025).

Future work is anticipated along several axes: (1) broader dataset construction beyond static images; (2) improved reward modeling with specialized models for consistency and identity; (3) diffusion-based detokenization to address text-rendering failures; and (4) advancing curriculum learning strategies for edit complexity vs. fidelity trade-offs (Tian et al., 18 Nov 2025).

GPT-IMAGE-EDIT-1.5M, with its public availability, refined pipeline, and benchmarking suite, constitutes a foundation for state-of-the-art research in large-scale, instruction-driven, and multimodal image editing.