ImageDoctor: Unified T2I Evaluation

• ImageDoctor is a unified multi-aspect evaluation framework for text-to-image generation that replaces single-scalar judgments with four-dimensional scoring on plausibility, semantic alignment, aesthetics, and overall quality.
• It integrates pixel-level artifact and misalignment heatmaps within a 'look-think-predict' diagnostic process, offering spatial flaw localization and clear, structured reasoning.
• Leveraging supervised fine-tuning and reinforcement learning, ImageDoctor improves human preference alignment and optimization signals by up to 10% over scalar-based reward models.

ImageDoctor is a unified multi-aspect evaluation framework for text-to-image (T2I) generation that replaces single-scalar judgment with four-dimensional scoring, pixel-level flaw localization, and explicit reasoning. It evaluates generated images along plausibility, semantic alignment, aesthetics, and overall quality, and it additionally produces artifact and misalignment heatmaps that can be reused as dense rewards for preference alignment. The framework is built on a vision-LLM, organized around a “look-think-predict” diagnostic process, and trained with supervised fine-tuning and reinforcement learning. In reported experiments, it shows stronger alignment with human preference than prior evaluators and, when used for preference tuning, yields a 10% improvement over scalar-based reward models (Guo et al., 1 Oct 2025).

1. Problem setting and evaluative scope

ImageDoctor is motivated by a specific limitation in T2I evaluation: existing approaches typically quantify the quality of a generated image using a single scalar. The framework instead treats image quality as multi-aspect and spatially grounded. Its four assessment dimensions are plausibility, semantic alignment, aesthetics, and overall quality. Plausibility measures realism and visual fidelity, including whether the image contains natural or distorted, implausible details; semantic alignment measures correspondence to the input prompt; aesthetics evaluates color use, composition, and overall visual appeal; and overall quality provides a holistic subjective rating. For each dimension, ImageDoctor produces a scalar score, typically between 0 and 1, together with a succinct rationale (Guo et al., 1 Oct 2025).

A second defining element is dense spatial feedback. ImageDoctor predicts two pixel-level heatmaps: an artifact heatmap for unrealistic, corrupted, or implausible elements, and a misalignment heatmap for regions that do not faithfully represent the prompt. This establishes a hybrid evaluation regime in which global judgment and local diagnosis are produced by a single model. In practical terms, that design makes the framework relevant not only for benchmarking and re-ranking, but also for downstream optimization of T2I generators.

This formulation also addresses a recurrent misconception in image preference modeling: that human judgment can be adequately compressed into a single reward value. ImageDoctor’s design explicitly rejects that assumption by pairing scalar outputs with grounded visual evidence and natural-language justification.

2. Architecture and the “look-think-predict” paradigm

The framework is built on a multimodal LLM, exemplified in the reported implementation by Qwen2.5-VL-3B. Scalar scores are generated directly by the MLLM, while heatmaps are produced by a dedicated decoder that leverages task tokens and visual embeddings. The decoder is inspired by the Segment Anything Model mask decoder and uses bi-directional cross-attention between task tokens and image embeddings, followed by upsampling to predict heatmaps at original image resolution. Special tokens, <ART> for artifacts and <MIS> for misalignment, specify which flaw type is being decoded (Guo et al., 1 Oct 2025).

Its central procedural abstraction is “look-think-predict.” In the “look” stage, the model first localizes regions likely to contain flaws, operationalized through predicted bounding boxes over problematic areas. In the “think” stage, it produces structured reasoning that synthesizes visual observations with prompt details and explains why a region is flawed or satisfactory. In the “predict” stage, it outputs the final scores, heatmaps, and a compact answer summary. The paper describes this as a human-inspired diagnostic process intended to improve detail sensitivity and reasoning capability.

The architectural coupling between reasoning and localization is important. Heatmap prediction is not treated as a purely low-level segmentation task; it is conditioned by semantic reasoning and prompt understanding. Conversely, score prediction is not purely abstract: it is informed by explicit localization of defects. This coupling is one of the main differences between ImageDoctor and earlier scalar-only evaluators.

3. Training procedure, supervision, and datasets

Training proceeds in multiple stages. In supervised fine-tuning, the MLLM backbone is first finetuned to predict scores for image-prompt pairs. A second supervised stage uses curated chain-of-thought data in “look-think-predict” format. The chain-of-thought data includes bounding boxes extracted from ground-truth heatmaps and reasoning chains generated with the help of Gemini 2.5 Flash and high-quality prompts. The model is then supervised to generate flaw boxes, rationale text, and final scores jointly. The reported supervised objective combines autoregressive text loss with an $L_2$ heatmap term:

$$\mathcal{L} = -\sum_{i} \log p_\theta(y_i \mid y_{<i}, I, P) + \sum_{d} \| M_d - \tilde{M}_d \|_2^2$$

where $I$ and $P$ denote the input image and prompt, and $M_d$ and $\tilde{M}_d$ are the ground-truth and predicted heatmaps for each task (Guo et al., 1 Oct 2025).

A reinforcement fine-tuning stage then applies Group Relative Policy Optimization (GRPO). For each input, the model generates multiple candidate “look-think-predict” responses. Each response receives a composite reward formed from grounding reward, score reward, and heatmap reward:

$$\mathcal{R} = \mathcal{R}_G + \mathcal{R}_S + \mathcal{R}_H$$

The group-normalized advantage is

$$A^i = \frac{\mathcal{R}^i - \text{mean}(\mathcal{R}^i)}{\text{std}(\mathcal{R}^i)}$$

and this is used in a clipped policy-gradient objective. The heatmap decoder is additionally optimized with pixel-level objectives including MSE, KLD, CC, and related losses.

The primary experimental dataset is RichHF-18K, described as a large-scale, multi-aspect human-annotated set with four-dimensional scores and heatmaps. Cross-dataset generalization is evaluated on GenAI-Bench and TIFA. This dataset choice is significant because ImageDoctor is trained and assessed not only on scalar preference labels but on localized flaw annotations and structured rationales, which directly support its grounded evaluation objective.

4. Reported empirical performance

On RichHF-18K score prediction, ImageDoctor outperforms the listed baselines across all four dimensions. Its reported PLCC values are 0.73 for plausibility, 0.68 for aesthetics, 0.81 for semantic alignment, and 0.75 for overall quality, for an average PLCC of 0.74. The strongest previously listed baseline in the table, RichHF, reports 0.69, 0.60, 0.47, and 0.58 respectively, with an average PLCC of 0.59. The gain is particularly pronounced on semantic alignment, where ImageDoctor reaches 0.81 against the 0.47–0.40 range reported for RichHF and CLIP (Guo et al., 1 Oct 2025).

For heatmap prediction on the same benchmark, ImageDoctor also improves the reported localization metrics. On artifact heatmaps it achieves CC of 0.57 and KLD of 1.48, compared with RichHF at CC 0.56 and KLD 1.65. On misalignment heatmaps it reports CC 0.23 and KLD 2.86, compared with RichHF at CC 0.21 and KLD 2.93. The paper further states that ImageDoctor outperforms all compared preference models on GenAI-Bench and TIFA, indicating cross-dataset generalization.

Ablation studies attribute these gains to both explicit localization and explicit reasoning. Removing either the “look” or the “think” stage degrades performance; the “look” stage is reported as especially important for heatmaps, and the “think” stage for score accuracy. Task tokens also improve heatmap decoder performance. These results indicate that the framework’s gains are not reducible to a stronger backbone alone; they depend on the joint formulation of localization, explanation, and scoring.

5. ImageDoctor as a dense reward model

ImageDoctor is not only an evaluator. The framework is also used as a reward model for reinforcement learning in T2I systems. The paper contrasts conventional Flow-GRPO, which uses image-level scalar rewards such as PickScore, with DenseFlow-GRPO, which leverages ImageDoctor’s pixel-level heatmaps to create dense, region-aware rewards. The stated motivation is that scalar rewards are coarse and sparse, whereas localized reward maps can target specific defective regions during optimization (Guo et al., 1 Oct 2025).

On DrawBench, the reported numbers show a consistent improvement from scalar to dense reward usage. The base model has ImageReward 0.82, CLIPScore 0.95, and UnifiedReward 2.90. PickScore reaches 1.00, 0.94, and 2.94. RichHF reaches 0.88, 0.94, and 2.92. ImageDoctor used as a scalar reward reaches 1.03, 0.96, and 2.96. ImageDoctor with DenseFlow-GRPO reaches 1.10, 0.97, and 3.00. The paper summarizes this as a further boost of up to 10% compared to the strongest prior scalar-based reward.

The significance of this result is methodological as much as empirical. ImageDoctor turns evaluation outputs into training signals without introducing a separate dense supervision pipeline. In this sense, the framework occupies both sides of the preference-alignment loop: it diagnoses outputs and also supplies localized gradients for improving future outputs.

6. Relation to adjacent diagnostic frameworks and representation debates

ImageDoctor belongs to a broader shift from scalar image preference to localized feedback. DiffDoctor, for example, also proposes a two-stage “diagnose-then-treat” pipeline, but it focuses on artifact reduction in image diffusion models through a robust artifact detector trained on over 1 million samples and then uses pixel-level artifact confidence maps to optimize the generator. Its treating loss directly minimizes artifact confidence at each pixel:

$$\mathcal{L}_{\text{pixel}(\theta)} = \frac{1}{h \times w} \sum_{i,j} C\left(\pi_\theta(z_T)\right)[i,j]$$

This establishes a close conceptual parallel with ImageDoctor’s dense reward use, although DiffDoctor is framed around binary artifact segmentation rather than four-dimensional human preference evaluation (Wang et al., 21 Jan 2025).

A later line of work explicitly critiques heatmap-centric diagnosis, including ImageDoctor, as a representation bottleneck. Structured Defect Grounding (SDG) argues that heatmaps formulate diagnosis as pixel-field regression and therefore struggle to localize variable-cardinality defects and bind semantic reasons to individual failures. SDG instead models each defect as a tuple of location, type, reason, and importance:

$\{(b_i, t_i, r_i, s_i)\}_{i=1}^K$

and uses this structured representation for evaluation and for BoxFlow-GRPO alignment. In that comparison, heatmap methods are described as giving per-pixel or per-region scalar signals, not separating or describing defect instances, and lacking semantic explanatory power for “what,” “why,” and “priority” (Zhang et al., 4 Jun 2026).

This suggests that ImageDoctor marks an important intermediate stage in the evolution of T2I diagnosis: richer and more interpretable than single-scalar preference models, yet still less instance-explicit than later structured grounding approaches. Its contribution lies in showing that a unified evaluator can jointly produce multi-aspect scores, spatial defect maps, and usable optimization signals within one vision-language framework.