ImageCritic: Reference-Guided Post-Editing

• ImageCritic is a framework that detects and corrects fine-grained inconsistencies in generative images by using high-quality reference images.
• It leverages attention disentanglement and a detail-binding encoder to address localized issues such as text errors, logo corruption, and misplacement of key elements.
• The system employs a modular, agent-based multi-round correction pipeline that integrates reference-degraded-target triplets to refine image details accurately.

ImageCritic is a reference-guided post-editing framework designed to automatically detect and correct fine-grained inconsistencies in images produced by modern generative models, particularly in scenarios where a reference image is available to specify target content at the detail level (Ouyang et al., 25 Nov 2025). Unlike prior approaches that primarily address global structural fidelity, ImageCritic targets failures in local detail preservation—such as text errors, logo corruption, and character misplacement—by leveraging explicit attention disentanglement mechanisms, a detail-binding encoder, and an agent-based correction pipeline. The following sections provide a detailed technical account of its composition, experimental efficacy, and design rationale.

1. Motivation: Fine-Grained Inconsistencies in Image Generation

Reference-guided generative models (e.g., DiT, UNet+attention architectures) frequently succeed at maintaining global layout and semantics but exhibit characteristic local inconsistencies. Observed error modes include text blurring, incorrect symbol rendering, mismatched characters, and spatial drift of distinctive elements. The root causes traced to VAE encoding/decoding imprecision, shallow feature dropout, and entangled multi-modal attention incapable of distinguishing reference-driven constraints from general input noise responses. Such issues are especially prominent in domains where local textual, symbolic, or branding fidelity is critical (e.g., product design, scene personalization).

To directly model these failure patterns, ImageCritic adopts a reference–degraded–target triplet methodology: the reference ($I_{\rm ref}$) is a high-quality target image, the degraded version ($I_{\rm deg}$) is generated by explicit, localized corruption of $I_{\rm tgt}$, and training proceeds on mapping $(I_{\rm ref}, I_{\rm deg}) \to I_{\rm tgt}$. This formulation compels the model to learn corrective inference grounded in authentic, fine-grained generative inadequacies.

2. Data Construction: Reference–Degraded–Target Triplets

Automated large-scale dataset construction is implemented via a multistage pipeline:

• VLM-based Selection and Annotation:
  • Crawl high-quality product and scene images.
  • Generate candidate variants using state-of-the-art text-to-image systems (Flux-Kontext, GPT-4o, Nano-Banana).
  • Filter variants for basic clarity/readability using Qwen-VL.
  • Generate object masks using Grounded-SAM, verified via Qwen-VL for object identity consistency.
• Explicit Degradation Protocol:
  • For each verified mask $M$, a patch (20–70% of mask area) is selected.
  • Corruption is injected via Flux-Fill conditioned on domain prompts (“English words,” “Chinese characters,” “logos,” or empty/clean).
  • The degraded image is $I_{\rm deg} = M\odot D + (1-M)\odot G$, where $D$ is the inpainted (corrupted) region and $G$ is the original.
• Statistics:
  • 10,000 triplets.
  • Degradation ratios are uniform in [20%, 70%].
  • Patch types: English text (30%), Chinese text (30%), logos (20%), empty/generic (20%).

Training on these curated triplets simulates and addresses the concrete error manifold found in contemporary generative outputs.

3. Model Architecture: Attention Alignment and Detail Binding

ImageCritic extends a DiT-style diffusion transformer (specifically, Flux-Kontext DiT) by integrating two principal modules:

• Detail Encoder:
  • Each image input ("IMG1" for reference, "IMG2" for degraded) is coupled with its corresponding text trigger embedding.

  • For each image $i \in \{R, I\}$, extract token $P_i$ from T5, CLIP embedding $C_i$; concatenate and project via a two-layer MLP with ReLU:

    $$P_i' = [P_i; C_i] \quad\to\quad \tilde{P}_i = \mathrm{MLP}(P_i')$$

  The updated embedding $\tilde{P}_i$ replaces the original trigger token, ensuring correct correspondence between image identity and textual prompt.

• Attention Alignment in DiT Blocks:

  • Double-stream attention blocks process noisy target tokens, text tokens, and control tokens.

  • Multi-head attention is formulated as

    $$\mathrm{Attn}(Z) = \mathrm{softmax}\left(\frac{QK^\top}{\sqrt d}\right)V, \quad Z=[z_{tgt},c_T,z_c]$$

  • Custom loss terms explicitly separate reference and input attention to minimize cross-entanglement (see below).

Inference orchestration uses a Qualifier Agent to crop defective regions and auto-prompt targeted guidance, for example: "Use the {object} in IMG1 as reference to correct IMG2."

4. Attention Alignment Loss: Disentangling Reference and Input Cues

The model's attention disentanglement is enforced through a joint loss on selected transformer layers. For each double-stream attention block $j$:

• Attention maps $M^j_R$ and $M^j_I$ represent the reference-to-noise and input-to-noise flows.
• Given binary mask $B$ (background=1, subject=0), and its complement $\bar B$, after min–max normalization $N(\cdot)$:

  $$L_G = \frac{1}{n_l} \sum_{j=1}^{n_l} \| B \odot N(M^j_I) \|_2^2$$

  $$L_R = \frac{1}{n_l} \sum_{j=1}^{n_l} \| \bar{B} \odot N(M^j_R) \|_2^2$$

The loss ensures input tokens focus on global/background context, while reference tokens specialize to the subject region. The total training objective is:

$\mathcal{L} = L_{\text{diff}} + L_G + L_R$

where $L_{\text{diff}}$ is the rectified flow-matching loss of the diffusion network.

Empirical ablation (see Section 8) demonstrates that each module (Detail Encoder and Attention Alignment Loss) independently yields measurable improvements, and their combination is additive for all similarity metrics.

5. Agent Framework: Multi-Round Local Inconsistency Correction

A modular agent-based system is implemented for closed-loop correction:

1. Inconsistency Detector (LLM): Receives (Reference, Target), outputs bounding box for most severe defect.
2. Reference Finder (LLM): Localizes clean source patch and generates object tags.
3. TagGrounder: Maps user-supplied tag to bounding box within Reference.
4. ImageCritic Model: Invoked for localized correction given (Reference, Degraded, Prompt).
5. Coordinator (Qwen-Agent): Orchestrates steps, integrates user feedback, and repeats for further rounds as required.

This structure enables multi-stage, user-in-the-loop refinement until the correction is validated as satisfactory. The approach is particularly suited to scenarios where generation artifacts are localized rather than structurally global.

6. Experimental Evaluation and Ablation

Benchmarks and Metrics

• Test suites: DreamBench++ (human-aligned personalization), CriticBench (focusing on fine localized artifacts, e.g., 200 product images, 100 apparel/accessory images).
• Baselines: Closed-source (Sora, GPT-4o, Nano-Banana); open-source (XVerse, DreamO, MOSAIC, OmniGen2, UNO, Qwen-Image).
• Metrics:
  • CLIP Image Similarity (CLIP-I, higher better)
  • DINO Score (higher better)
  • DreamSim (lower better)

Method	CLIP-I	DINO	DreamSim
Best baseline (avg.)	76.5	--	0.0
ImageCritic (avg.)	79.9	--	–1.7

Main improvements on CriticBench for all baselines, e.g., CLIP-I +1.3 pts (up to +3.4 for XVerse), DINO +1.2 pts, DreamSim –1.7 pts.

Ablation Study

AAL	DE	ΔCLIP-I	ΔDINO	ΔDreamSim
–	–	+0.0	+0.0	0.0
✓	–	+0.7	+0.9	–0.9
–	✓	+0.7	+0.7	–0.9
✓	✓	+1.3	+1.2	–1.7

AAL (Attention Alignment Loss) and DE (Detail Encoder) are complementary: AAL disentangles attention; DE ensures correct text-image binding.

7. Limitations and Future Directions

• Scope: ImageCritic excels at localized, reference-driven correction; it is less effective where global content or geometry must be re-imagined.
• Dependency on Mask Accuracy: Performance can be degraded by failures in SAM or VLM mask generation.
• Future Work:
  • Extending to global geometry/style corrections using higher-level alignment objectives.
  • Expanding triplet data mining to broader classes (faces, scenes) using self-supervised signals.
  • Integrating with downstream tasks (e.g., OCR, visual QA) for end-to-end feedback and robustness.

---

ImageCritic exemplifies a targeted approach for post-hoc correction and evaluation of generated images using explicit reference guidance, attention disentanglement, and agent-based editing, with demonstrated improvements over both open- and closed-source state-of-the-art baselines across multiple human-aligned and fine-detail benchmarks (Ouyang et al., 25 Nov 2025).