---
title: 'GPT-IMAGE-EDIT-1.5M: Image Editing Benchmark'
url: https://www.emergentmind.com/topics/gpt-image-edit-1-5m
type: topic
---

# GPT-IMAGE-EDIT-1.5M: Image Editing Benchmark

GPT-IMAGE-EDIT-1.5M is a large-scale, instruction-driven image-editing corpus and research benchmark. It comprises over 1.5 million high-quality triplets—{instruction, source image, edited image}—systematically curated and refined to advance the development and evaluation of instruction-following image editing models. This dataset forms the backbone for state-of-the-art open-source models and unified multimodal architectures, narrowing performance gaps with proprietary systems.

## 1. Dataset Definition and Source Composition

GPT-IMAGE-EDIT-1.5M consists of more than 1.5 million data points, each a triplet containing: (i) a natural-language editing instruction, (ii) a source image, and (iii) a corresponding edited image strictly executing the instruction. Images are provided at fixed aspect ratios—$1{:}1$ (1024×1024), $3{:}2$ (1536×1024), and $2{:}3$ (1024×1536)—with approximate distribution of 40%, 30%, and 30% across the corpus [2507.21033]. The dataset unifies and refines three major open image-editing datasets: OmniEdit, HQ-Edit, and UltraEdit, chosen for their variety of edit types but previously limited by either noisy instructions or suboptimal visual fidelity.

Edit operations are broadly distributed as follows: add/remove/replace subject (~25%), background change (~12%), color/texture adjustment (~12%), style transfer (~10%), text insertion (~5%), tone/lighting (~8%), composite/complex edits (≥2 atomic ops, ~15%), and other categories (e.g., motion, portrait tweaks) covering the remaining spectrum [2507.21033].

## 2. Data Construction and Quality Control Pipeline

The construction pipeline involves three principal stages: output image regeneration, instruction rewriting, and full input–output pair regeneration where necessary. 

**a) Output Regeneration:** For each original {instruction, source image} pair, the system queries GPT-4o’s “gpt-image-1” API, generating higher-fidelity edited images. All outputs adhere to one of the specified resolutions and undergo artifact filtering—specifically, padding artifacts must not exceed 0.5% residual border, and any generation failures are culled automatically. Upscaling/downscaling are applied to harmonize image sizes from all source datasets [2507.21033].

**b) Instruction Rewriting:** To improve semantic alignment, 10% of instructions (OmniEdit subset) are rewritten by GPT-4o, conditioned on the new output and the source image, further increasing instruction adherence [2507.21033].

**c) Full Pair Regeneration:** For 50% of HQ-Edit, both the source and edited images are newly synthesized by GPT-4o: first generating a source image given the instruction, then reapplying the instruction to that generated source for the edit pair [2507.21033].

All procedures are automated, with no manual annotation beyond automated checks. Filtering leverages learned scoring models and thresholds like CLIPScore ≥ 0.35, area coverage for object/background edits, and artifact/fake detection via dedicated models [2505.20275].

## 3. Metadata Schema and Benchmarks

Each data sample includes structured metadata such as references to source and edited images, object masks, bounding boxes, resolution, natural-language caption, edit instruction, edit category/type, and automated quality scores. JSON schema is standardized:

```json
{
  "id": "IMG00012345",
  "ref_image": "images/ref/IMG00012345.jpg",
  "edited_image": "images/edit/Remove/IMG00012345_edit.jpg",
  "mask": "masks/Remove/IMG00012345_mask.png",
  "resolution": [1600, 2560],
  "caption": "A tabby cat sitting on a wooden deck.",
  "instruction": "Remove the plastic bottle lying next to the cat.",
  "edit_type": "Remove",
  "bbox": [400,1200,600,1400],
  "post_score": 4.7,
  "fake_score": 0.05
}
```
[2505.20275]

Benchmarks built on GPT-IMAGE-EDIT-1.5M leverage several standardized suites: GEdit-EN (11 edit types, instruction adherence, perceptual quality), ImgEdit-Full (9 task families), and Complex-Edit (three atomic edits per instruction for 50% of OmniEdit). Metrics include MLLM-based scoring for instruction following and perceptual quality, CLIPScore, FID, LPIPS, and composite evaluation for complex-editing performance [2507.21033].

## 4. Model Development, Training Paradigms, and Downstream Performance

Fine-tuning on GPT-IMAGE-EDIT-1.5M enables substantial improvements over previous open-source data regimes. Notably, FLUX.1 Kontext dev (flow-matching backbone) with Qwen-VL-7B and T5 text encoders achieves highly competitive results: GEdit-EN score 7.24, ImgEdit-Full 3.80, Complex-Edit overall 8.78, closely matching proprietary GPT-4o scores [2507.21033]. 

The canonical architecture (as in UniGen-1.5 [2511.14760]) leverages a 7B parameter transformer backbone, frozen SigLIP2 continuous encoders and discrete MAGViTv2 tokenizers for vision input, and a three-way condition concatenation strategy—continuous vision, text instruction, and discrete vision. Training proceeds through pre-training on multi-million image–text pairs, supervised finetuning on synthetic generation and editing pairs, a dedicated instruction alignment phase, and RLHF via Group Relative Policy Optimization (GRPO) with shared reward models [2511.14760].

Table: Benchmark Comparison (ImgEdit-Full)

| Model        | Overall Score |
| ------------ | ------------ |
| Ours (GPT-IMAGE-EDIT-1.5M → FluxKontext) | 3.80          |
| GPT-4o       | 4.20          |
| Best previous open-source    | 4.00          |
[2507.21033]

## 5. Unified RLHF and Reward Optimization

The UniGen-1.5 approach shares a unified RL objective for text-to-image generation and image editing, using a family of reward models: CLIP-ViT-H cosine, HPSv2 human-preference, UnifiedReward-7B, and an ORM (“Yes/No” outcome). The reward for candidate $\hat{H}$ conditioned on $T$ is:

$$
R(\hat{H}, T) = \frac{R_C + R_H + R_U + R_O}{4}
$$

Policy optimization employs the GRPO algorithm, calculating normalized groupwise advantages and penalizing deviation from a reference policy via KL divergence. Post-supervised finetuning, a lightweight “Edit Instruction Alignment” phase (≈17,000 triplets) further sharpens the model’s capacity to map instructions to edits, increasing the reward variation necessary for effective RL [2511.14760].

Ablation confirms that both unified RL across tasks and explicit instruction alignment substantively boost editing and generation scores, with compositional conditions ($\mathcal{X}_C^U \rightarrow \mathcal{T}_C \rightarrow \mathcal{X}_C^G$) found optimal [2511.14760].

## 6. Corpus Structure, Complexity, Expansion, and Limits

The dataset includes ≈1.35M single-turn edits (basic operations, novel/complex categories) and 0.15M multi-turn tasks for content memory, understanding, and version backtracking. Each edit type and complexity level is precisely quota-managed to avoid bias, using mining from LAION-5B, video frame extraction (Open-Sora Plan), and synthetic prompt chaining for category coverage [2505.20275]. Automated judging and QA using large vision-language models (e.g., GPT-4o) provides per-sample scoring, with a subset (up to 10%) manually spot-checked.

Known limitations include reliance on proprietary models (e.g., GPT-4o) for output and instruction generation, degradation in realism/identity for high edit-complexity (C3+ in Complex-Edit), and persistent challenges in categories such as text insertion [2507.21033]. Proposed directions include extending to multimodal scenarios (video, 3D, multi-view), hybrid crowdsourcing pipelines for annotation, and more sophisticated misalignment detectors.

## 7. Impact and Future Directions

GPT-IMAGE-EDIT-1.5M demonstrably closes the gap between open-source and proprietary instruction-based image editing. The corpus’s scale, compositional diversity, operational rigor, and benchmarked performance underpin its adoption for both dataset building and model development [2507.21033, 2511.14760].

Future work is anticipated along several axes: (1) broader dataset construction beyond static images; (2) improved reward modeling with specialized models for consistency and identity; (3) diffusion-based detokenization to address text-rendering failures; and (4) advancing curriculum learning strategies for edit complexity vs. fidelity trade-offs [2511.14760].

GPT-IMAGE-EDIT-1.5M, with its public availability, refined pipeline, and benchmarking suite, constitutes a foundation for state-of-the-art research in large-scale, instruction-driven, and multimodal image editing.

Source: https://www.emergentmind.com/topics/gpt-image-edit-1-5m