---
title: 'MICON-Bench: Multi-Image Context Benchmark'
url: https://www.emergentmind.com/topics/micon-bench
type: topic
---

# MICON-Bench: Multi-Image Context Benchmark

MICON-Bench is a benchmark and evaluation framework for **multi-image context image generation** in **Unified Multimodal Models (UMMs)**, introduced in “MICON-Bench: Benchmarking and Enhancing Multi-Image Context Image Generation in Unified Multimodal Models” by Mingrui Wu, Hang Liu, Jiayi Ji, Xiaoshuai Sun, and Rongrong Ji, with affiliations including Xiamen University and Zhongguancun Academy [2602.19497]. It is designed to measure and enhance a setting in which a model receives two or more related reference images together with a textual instruction and must synthesize a single image that is semantically aligned and visually consistent with those references. The framework combines a six-task dataset, an **MLLM-driven Evaluation-by-Checkpoint** verifier, and a training-free inference mechanism called **Dynamic Attention Rebalancing (DAR)**. Its stated focus is on **cross-image composition**, **contextual reasoning**, and **identity preservation**, with the explicit goal of exposing failure modes that are not systematically probed by benchmarks centered on text-to-image generation or single-image editing [2602.19497].

## 1. Problem setting and scope

Multi-image context image generation asks a model to integrate and reason across multiple visual references while obeying a textual instruction. The task demands include preserving identities and attributes taken from specific images, composing foregrounds and backgrounds across references, arranging multiple entities under spatial constraints, transferring components and styles, and maintaining cross-reference consistency [2602.19497].

The paper distinguishes this setting from two adjacent problem classes. Traditional text-to-image generation grounds only on a single text prompt, whereas single-image editing modifies one image conditioned on text or an auxiliary mask. By contrast, multi-image context generation requires a model to fuse and disambiguate information from multiple visual sources, reason about inter-image relationships such as identity, attributes, and geometry, and produce one coherent output that respects constraints drawn from each reference [2602.19497].

The motivation for a new benchmark is framed against existing benchmarks such as Geneval, T2ICompBench, ImgEdit-Bench, and EMU Edit, which are described as largely targeting text alignment, realism, or single-image faithfulness rather than systematic evaluation of cross-image composition, multi-reference consistency, or contextual narrative reasoning. MICON-Bench is therefore positioned as a direct probe of three core capabilities: whether a model can correctly combine subjects and backgrounds across references without contradictions, whether it can follow spatial relations, causal logic, and story progression implied by multi-image inputs, and whether it can preserve instance-level identity and fine-grained attributes from specific references despite distractors [2602.19497].

A plausible implication is that the benchmark is not merely an expanded prompt suite but an attempt to isolate failure modes specific to **multi-source visual grounding**, especially when identity, geometry, and causality must be inferred jointly.

## 2. Benchmark composition

MICON-Bench contains **1,043 cases** and **2,518 images** spanning six tasks. The tasks are designed to stress distinct but overlapping forms of multi-image reasoning, from subject-scene composition to part-level transfer and narrative continuation [2602.19497].

| Task | Core objective | Cases / images |
|---|---|---|
| Object Composition | Combine one or more subjects with a background scene while preserving identities and salient attributes | 200 / 482 |
| Spatial Composition | Arrange multiple objects under explicit geometric relations with identity preservation | 200 / 498 |
| Attribute Disentanglement | Recombine subject, style, and background from three distinct references | 100 / 300 |
| Component Transfer | Extract specific elements from one reference and apply them to a target subject in another | 240 / 601 |
| FG/BG Composition | Extract the foreground from one reference and combine it with the background of another | 200 / 400 |
| Story Generation | Infer and render what happens next from two or three reference panels | 103 / 237 |

Each task is defined in terms of a target relationship among references.

**Object Composition** uses 2 or 3 reference images depicting a subject or a scene and asks for one synthesized image containing all specified subjects in the designated environment. The representative prompt is: “Generate an image that contains both the complete {obj_a} and {obj_b} together in {chosen_scene}.” For three references, the formulation becomes “all three objects ({obj_a}, {obj_b}, and {obj_c}) together in {chosen_scene}.” The relationship of interest is **subject-to-scene composition** with **identity and attribute fidelity** [2602.19497].

**Spatial Composition** extends composition by imposing explicit geometric constraints such as left, right, or front. The representative prompt is: “Generate an image that contains both the complete {obj_a} and {obj_b} together in {chosen_scene}, with {obj_a} positioned to the {spatial_relation} of {obj_b}.” For three references, the prompt specifies a left-center-right arrangement. The targeted relationship is **multi-object spatial geometry** under **physical plausibility** [2602.19497].

**Attribute Disentanglement** uses three references, labeled A, B, and C, where A provides the subject, B the style, and C the background. The representative prompt is: “Generate an image of the {main_object} from image A, using the visual style from image B, and placing it in the {specific_background} environment from image C.” The task probes **cross-image attribute disentanglement and transfer with identity fidelity** [2602.19497].

**Component Transfer** focuses on part-level control. It asks the model to extract specific elements such as clothing, accessories, or parts from one reference and apply them to a target subject in another. The benchmark defines both a complex mode and a simple mode, with representative prompts including “Extract {elements_desc} from {source_desc} in Image {source_label}, then apply these elements to {target_desc} in Image {target_label}” and “Extract only the {local_element} from the subject in Image A, then apply this element to the subject in Image B.” This targets **fine-grained identity and attribute control** [2602.19497].

**FG/BG Composition** isolates foreground-background separation. It uses two references and asks the model to cleanly extract the foreground from one image and blend it into the unchanged background of the other. The representative prompt is: “Generate an image where you cleanly extract the {obj_a} from image A and replace the {obj_b} in image B. The background from image B should remain unchanged.” The targeted relationship is **foreground-background separation** with **seamless blending** [2602.19497].

**Story Generation** differs from the other five tasks by requiring causal and temporal inference. It uses two or three reference panels from four-panel comics generated by GPT-4o-Image and Nano-Banana, with human-filtered story texts and DiffBIR enhancement for clarity. The representative prompt is: “Given the reference images, infer and generate a realistic photo of what might happen next.” The relevant relationship is **cross-image causal reasoning**, **temporal consistency**, and **narrative closure** [2602.19497].

## 3. Data construction and curation

For the five compositional tasks, the reference images are generated via **Qwen-Image** from carefully constructed pools of subjects, attributes, spatial relations, and scenes using aligned prompt templates [2602.19497]. For **Story Generation**, short narratives are LLM-generated and human-filtered; reference panels are then produced by **GPT-4o-Image** and **Nano-Banana**, restored with **DiffBIR**, and filtered by humans for narrative and visual consistency [2602.19497].

The curation procedure uses a **two-stage filtering pipeline** that merges **MLLM coarse verification** with **meticulous human review** to ensure fidelity and diversity. The paper does not specify train/val/test splits and states that the benchmark is designed for evaluation. It also states that licensing details are not explicitly specified in the paper and should be checked in the repository [2602.19497].

Per-task reference cardinalities are explicit. Object Composition has **118** cases with 2 references and **82** with 3 references. Spatial Composition has **102** with 2 references and **98** with 3 references. Attribute Disentanglement uses **3 references in all 100 cases**. Component Transfer has **119** cases with 2 references and **121** with 3 references. FG/BG Composition uses **2 references in all 200 cases**. Story Generation has **72** cases with 2 references and **31** with 3 references [2602.19497].

These design choices indicate that MICON-Bench is structured around controlled compositional factors rather than unrestricted web-scale collection. This suggests an emphasis on **diagnostic benchmarking**: the benchmark aims to separate errors in identity binding, attribute transfer, spatial arrangement, and causal continuation rather than to maximize ecological diversity.

## 4. Evaluation-by-Checkpoint

The evaluation framework is **MLLM-driven**. For each case, the instructions, the reference images, and the model’s generated image are fed into an MLLM verifier, with **Qwen3-VL-32B-Instruct** as the default. The verifier checks task-specific binary checkpoints grouped under seven dimensions: **Instruction Following**, **Identity/Fidelity**, **Structure/Geometry**, **Cross-Reference Consistency**, **Causality**, **Text Grounding**, and **Overall Usability** [2602.19497].

Each checkpoint yields pass or fail, and the proportion of satisfied checkpoints within a dimension is converted to a score in $[0,1]$. One **hard-constraint checkpoint** per dimension can cap that dimension’s score at **0.4** if failed. The active dimension scores are then aggregated and linearly rescaled to **$[0,100]$** [2602.19497].

For **Story Generation**, the benchmark combines checkpoint scoring with matching against a human-authored answer set. The paper gives the formula
$$
S_{\text{story}} = 0.4 \cdot S_{\text{checkpoint}} + 0.6 \cdot S_{\text{answer}}.
$$
This weighting is intended to ensure both causal progression and narrative plausibility [2602.19497].

The rubric assigns distinct semantic roles to the dimensions. **Identity/Fidelity** emphasizes instance-level identity, category correctness, and key attribute preservation such as color, texture, and shape. **Structure/Geometry** verifies spatial relations and physical plausibility. **Cross-Reference Consistency** checks contradictions across references. **Causality** verifies temporal logic. **Text Grounding** checks textual content. **Overall Usability** measures naturalness and coherence [2602.19497].

The paper also notes that checkpoint questions may be phrased generically or specifically, for example “Does the image include all specified objects?” versus “Does the image include a giraffe and a wooden chair?”, and reports stability across phrasing via a prompt sensitivity study. Reliability is evaluated on a sampled **120-case subset** using **three expert annotators** with the same binary hard-constraint rubric. The automated verifier’s **average deviation from human scores is +0.67**, and model rankings are reported as consistent. Replacing Qwen3-VL with **InternVL3.5-38B** on the full dataset preserves relative rankings, and five independent runs evaluating **BAGEL** with Qwen3-VL differ by at most **0.04** [2602.19497].

A central limitation is also explicitly stated: as with any MLLM-based evaluation, verifier hallucinations can propagate, and the framework’s reliability depends on the MLLM’s perception. The paper argues that hard constraints and binary checkpoints improve robustness, but it does not treat verifier dependence as eliminated [2602.19497].

## 5. Dynamic Attention Rebalancing

**Dynamic Attention Rebalancing (DAR)** is a **training-free, plug-and-play** mechanism applied during inference. Its purpose is to adjust attention to reference image tokens so that UMMs focus on relevant regions across multiple references. The method identifies over-attended and under-attended areas via sampled attention maps and then reweights keys to amplify semantically relevant tokens and suppress distractors, with the stated effect of reducing cross-image hallucinations and improving identity and spatial coherence [2602.19497].

The query sampling rule is given as
$$
\left\{\left\lfloor i \cdot \frac{L_q - 1}{m - 1} \right\rfloor \right\}_{i=0}^{m-1}.
$$
The paper defines token relevance by
$$
r_k = \sum_{i=1}^{m} \sum_{h=1}^{H} \tilde{A}_{i,h,k},
$$
followed by min-max normalization,
$$
\hat{r}_k = \frac{r_k - \min_j r_j}{\max_j r_j - \min_j r_j}.
$$
The algorithmic description then thresholds $\hat{r}_k$ with $\tau_{\mathrm{high}}$ and $\tau_{\mathrm{low}}$ to assign weights $w_k \in \{1-\gamma, 1, 1+\gamma\}$, recomputes full attention from $Q$ to the reweighted keys $w \odot K_{\mathrm{ref}}$, and continues the generation process [2602.19497].

The operational steps are explicit:

1. Sample $m$ representative query tokens uniformly from $L_q$ using the stated indices.
2. Compute attention from sampled queries to reference keys across $H$ heads.
3. Sum and min-max normalize per reference token to obtain $\hat{r}_k$.
4. Threshold $\hat{r}_k$ with $\tau_{\mathrm{high}}$ and $\tau_{\mathrm{low}}$ to assign weights.
5. Recompute full attention using the reweighted keys and proceed with generation [2602.19497].

The default hyperparameters are $\gamma=0.15$, $m=64$, $\tau_{\mathrm{high}}=0.7$, and $\tau_{\mathrm{low}}=0.3$. On **Object Composition** with **A800-40GB GPUs**, DAR adds approximately **5–10% runtime**, with examples reported as **BAGEL 57.13s → 61.70s** and **OmniGen2 72.01s → 74.71s** for 2 references [2602.19497].

DAR is applied to the open-source UMMs **BAGEL** and **OmniGen2**, and is reported to generalize on **OmniContext** and **XVerseBench**. The paper characterizes it as conceptually compatible with other UMMs that expose cross-attention over image tokens [2602.19497].

Ablation results show that performance declines as the number of references increases; one cited example is **BAGEL Object task: 2 refs 88.50 → 5 refs 66.36**, highlighting the challenge of multi-source fusion. The weight factor $\gamma=0.15$ is reported to provide the best overall trade-off, whereas excessively large $\gamma$ harms robustness; the paper notes that **$\gamma=0.55$ yields very low accuracies**. Qualitatively, DAR is reported to suppress spurious attention on irrelevant background regions or subjects and to improve localization of intended reference entities [2602.19497].

## 6. Empirical findings, usage, and limitations

The benchmark evaluates several model categories: proprietary or native UMMs (**Nano-Banana**, **GPT-Image**), diffusion-based systems (**UNO**, **DreamOmni2**, **Qwen-Image-Edit-2507**), and open-source UMMs (**BAGEL**, **OmniGen2**) together with their DAR-augmented variants [2602.19497].

On MICON-Bench with the **Qwen3-VL** verifier, **GPT-Image** achieves the highest reported average score at **91.51**, with task scores of **Object 96.45, Spatial 94.41, Attribute 93.39, Component 87.69, FG/BG 90.15, Story 85.99**. **Nano-Banana** reports **Object 95.60, Spatial 93.79, Attribute 92.13, Component 84.23, FG/BG 83.13, Story 82.84, Avg 89.25**. Among diffusion-based systems, **UNO** reports **Avg 44.76**, **DreamOmni2 75.56**, and **Qwen-Image-Edit-2507 72.96**. Among open-source UMMs, **BAGEL** improves from **73.55** to **76.31** with DAR, a gain of **+2.76**, while **OmniGen2** improves from **67.83** to **69.21**, a gain of **+1.38** [2602.19497].

The reported gains are task-dependent. DAR gains are described as notable in **Component** and **FG/BG** tasks for **OmniGen2**, and in **FG/BG** and **Story** for **BAGEL**. On **OmniContext**, DAR improves **BAGEL average 5.536 → 5.804** and **OmniGen2 7.532 → 7.768**. On **XVerseBench**, DAR improves **BAGEL overall 45.26 → 46.23** and **OmniGen2 51.14 → 51.73**, with consistent gains in identity- and attribute-similarity metrics [2602.19497].

For **Object** and **Spatial** tasks, the benchmark also reports auxiliary metrics. **CLIP T2I** changes from **0.3586/0.3748 → 0.3612/0.3761** for BAGEL with DAR, and from **0.3646/0.3828 → 0.3648/0.3828** for OmniGen2. The paper further states that **CLIP I2I**, **DINOv2 I2I**, and **LPIPS I2I** improve on most measures for BAGEL and OmniGen2, indicating stronger semantic alignment and visual quality [2602.19497].

The practical workflow is straightforward. The repository provides the dataset, prompt templates, evaluation scripts, and verifier configurations, with **Qwen3-VL-32B-Instruct** as default and **InternVL3.5-38B** supported. The stated procedure is: generate images for each benchmark case using the provided prompts and references; run the Evaluation-by-Checkpoint script to feed inputs, references, outputs, and checkpoints to the MLLM verifier; then obtain dimension-wise scores, apply hard-constraint capping and rescaling, and compute final aggregate scores, with Story using the **40%/60%** checkpoint/answer-set combination. Experiments were run with **4× NVIDIA A800 (40G)**, and the paper notes that the MLLM evaluator is stable across runs, with a maximum score discrepancy of **0.04** in repeated trials [2602.19497].

The limitations are explicit. Evaluation relies on an MLLM’s visual reasoning, so verifier hallucinations may bias scores even with hard constraints, binary checks, and human alignment. The tasks focus on image synthesis from multi-image inputs, leaving domains such as **video**, **3D**, and **long-range temporal narrative** open. DAR depends on the quality of the base attention maps; if a model misinterprets references or misses fine-grained semantics, reweighting may amplify incorrect signals. The paper identifies stronger calibrated verifiers with counterfactual tests, broader multi-image tasks, identity- and geometry-aware training objectives, attention diagnostics for interpretability, and scaling DAR to varied architectures and diffusion backends as future directions [2602.19497].

The release is centered on the public GitHub repository at **https://github.com/Angusliuuu/MICON-Bench**, which provides prompt templates, references, evaluation scripts, and verifier configurations. The paper states that exact usage terms and licensing should be verified in the repository itself [2602.19497].

Source: https://www.emergentmind.com/topics/micon-bench