AlignEvaluator: Fine-Grained T2I Reward Model
- AlignEvaluator is a dedicated reward model designed to assess prompt-image alignment by evaluating 24 key points across 6 super-categories.
- It transforms qualitative prompt fidelity into a scalar reward that drives Chain-of-Thought prompt rewriting, mitigating issues like attribute binding and spatial layout errors.
- Empirical integration into reinforcement learning pipelines demonstrates improved accuracy across multiple dimensions, ensuring generated images better meet semantic requirements.
AlignEvaluator is the dedicated reward model for fine-grained prompt-image alignment introduced in "PromptEnhancer: A Simple Approach to Enhance Text-to-Image Models via Chain-of-Thought Prompt Rewriting" (Wang et al., 4 Sep 2025). It evaluates a generated pair not as a generic aesthetic artifact or a broadly relevant caption match, but as an instance of whether a text-to-image system has actually satisfied the semantic requirements expressed in a prompt. In PromptEnhancer, it assesses each pair along 24 fine-grained key points organized into 6 super-categories, and returns a scalar reward used to optimize a Chain-of-Thought prompt rewriter. AlignEvaluator is therefore the mechanism that operationalizes prompt fidelity as a structured training signal rather than a coarse holistic score (Wang et al., 4 Sep 2025).
1. Definition and functional role
AlignEvaluator is defined as a model whose purpose is to provide an explicit training signal about whether a generated image satisfies the semantic requirements expressed in a prompt, rather than whether the image is merely attractive or broadly relevant to a caption (Wang et al., 4 Sep 2025). Its input is a text-image pair, and in the PromptEnhancer reinforcement-learning loop that pair is specifically written as , where is a rewritten prompt and is the image produced from it. Its output during policy training is a scalar reward (Wang et al., 4 Sep 2025).
This functional role is narrower and more specific than generic image-text similarity scoring. The paper repeatedly motivates AlignEvaluator by failures of text-to-image systems on attribute binding mistakes, negation, counting, spatial layout, complex interactions, and related mismatches between user intent and generated output. Prior reward mechanisms such as CLIP-based similarity, ImageReward, and other broad human-preference-like scores are described as providing only a coarse holistic signal; they may reflect overall relatedness or aesthetic preference, but they do not reveal what specifically went wrong in a rendering (Wang et al., 4 Sep 2025).
A common misconception is to treat AlignEvaluator as a generic judge of image quality. The paper presents the opposite design: AlignEvaluator is central precisely because it supplies explicit, structured, T2I-specific supervision. This suggests that its importance in PromptEnhancer lies less in absolute image assessment than in transforming prompt rewriting into a targeted optimization problem.
2. Taxonomy of alignment dimensions
The core of AlignEvaluator is a systematic taxonomy of 24 key points derived from common text-to-image failure modes and organized into six super-categories (Wang et al., 4 Sep 2025). The paper presents these dimensions as the evaluator’s internal basis for assessing prompt-image alignment.
| Super-category | Key points |
|---|---|
| Linguistic Comprehension | Negation; Attribute Consistency; Pronoun Resolution |
| Visual Attributes | Counting; Size; Material; Expression; Artistic Style |
| Action & Interaction | Full-body Action; Hand Action; Animal Action; Contact Interaction; Interaction w/o Contact; State |
| Relations & Structure | Comparative Relation; Compositional Relation; Containment Relation; Similarity Relation; Cross-Entity Binding; Entity Layout |
| World Knowledge & Reasoning | Knowledge Application; Counterfactual |
| Scene Text & Typography | Text Rendering; Text Layout |
Within Linguistic Comprehension, the paper gives examples such as Negation with “A bowl of beef noodles, no scallions,” Attribute Consistency with “Five people all wearing red clothes,” and Pronoun Resolution with “The large ball broke the table because it was made of metal,” where “it” refers to the ball (Wang et al., 4 Sep 2025). The paper’s rationale is that prompt-image alignment is not a single opaque property but a collection of recurring capability axes that map onto observed prompt-following errors.
The taxonomy spans both lower-level and higher-level phenomena. It includes simple failures such as Counting and Material, but also more difficult cases such as Cross-Entity Binding, Comparative Relation, Counterfactual, and Text Layout. The paper does not formally prove that these 24 dimensions are exhaustive, but it presents them as a “comprehensive” and “systematic” taxonomy sufficient to provide the explicit, multi-faceted reward needed for reinforcement learning (Wang et al., 4 Sep 2025). This suggests that AlignEvaluator is best understood as a structured evaluator whose interpretability depends directly on the taxonomy’s coverage.
3. Inputs, supervision, and unspecified internals
AlignEvaluator is trained on a large-scale dataset of pairs, each annotated with scores for the 24 key points (Wang et al., 4 Sep 2025). The paper is clear that the evaluator takes text-image pairs as input and that its job is to assess alignment between a prompt and a generated image. However, several implementation details are not disclosed.
The paper does not specify the exact annotation format for the 24 scores. It does not state whether these labels are binary, ordinal, continuous, pairwise, or category-specific pass/fail judgments. It also does not provide the dataset size for AlignEvaluator training, the exact annotation interface, or a detailed rubric beyond the taxonomy and examples. Although the PromptEnhancer pipeline mentions professional annotators in the supervised fine-tuning data pipeline, the paper does not explicitly describe a separate human annotation pipeline for AlignEvaluator labels themselves, nor whether any labels are generated synthetically or automatically (Wang et al., 4 Sep 2025).
The model architecture is also only partially specified. The paper calls AlignEvaluator a reward model, but does not identify a concrete backbone architecture, parameter count, encoder type, fusion mechanism, tokenizer, image resolution, or training recipe. What is specified is its I/O behavior: input , internal evaluation basis of 24 fine-grained key points, and scalar reward output during policy training (Wang et al., 4 Sep 2025). Likewise, the paper does not provide the loss used to train AlignEvaluator itself, nor a formal equation mapping 24 key-point scores into the scalar reward. A plausible implication is that the scalar reward must aggregate the fine-grained judgments, but the aggregation rule is not specified.
4. Role in PromptEnhancer’s reinforcement-learning pipeline
PromptEnhancer has a two-stage training pipeline in which AlignEvaluator is the reward source for the second stage (Wang et al., 4 Sep 2025). First, the CoT Rewriter is initialized by supervised fine-tuning on data. Second, the rewriter is aligned with reinforcement learning using Group Relative Policy Optimization (GRPO).
In this stage, the policy is the CoT Rewriter, initialized from Hunyuan-7B-Instruct and then SFT-tuned. For each original prompt, the policy samples candidate rewritten prompts: 0 These rewrites are passed to the frozen HunyuanImage 2.1 generator, which produces corresponding images 1. AlignEvaluator then computes a scalar reward 2 for each pair 3, and these rewards are used to rank the sampled rewrites and update the rewriter policy (Wang et al., 4 Sep 2025).
The paper is explicit that AlignEvaluator does not evaluate the rewritten prompt in isolation. It evaluates the alignment of the generated image with the rewritten prompt. This means the optimization is indirect: the reward is based on downstream image generation under the rewritten prompt, not on text-only similarity between the original and rewritten prompts (Wang et al., 4 Sep 2025). The paper does not mention an explicit secondary reward term for preserving original prompt semantics, though it notes that the SFT stage and a KL-divergence penalty with coefficient 4 regularize policy updates and prevent deviation from the SFT-initialized model.
The practical significance is that AlignEvaluator is the component that turns prompt rewriting from generic prompt expansion into reward-driven policy optimization. The paper also frames this as model-agnostic or generator-agnostic, because the rewriter is decoupled from the image generator and the base T2I model’s weights remain frozen (Wang et al., 4 Sep 2025). A common misunderstanding is to imagine AlignEvaluator as an inference-time reranker. The described pipeline instead indicates that it is used primarily during training as a reward model; the paper does not explicitly state that it is used at inference time for candidate selection or reranking.
5. Empirical evidence and what it does not show
The paper’s evidence for AlignEvaluator is indirect but substantial: it evaluates the full PromptEnhancer system whose second-stage reinforcement learning relies on AlignEvaluator (Wang et al., 4 Sep 2025). On the authors’ benchmark using HunyuanImage 2.1 as the base model, PromptEnhancer improves average accuracy across all 24 dimensions from 65.9% to 71.0%, a gain of 5.1 percentage points, with gains in 21 of 24 categories.
The strongest improvements occur in dimensions that the evaluator was explicitly designed to capture:
| Category | Change |
|---|---|
| Similarity Relation | +17.3 pp |
| Counterfactual | +17.2 pp |
| Counting | +15.0 pp |
| Pronoun Resolution | +13.9 pp |
| Expression | +12.9 pp |
| Cross-Entity Binding | +11.3 pp |
The paper also reports neutral or negative changes in a few categories: Contact Interaction at +0.0 pp, Artistic Style at +0.9 pp, Interaction w/o Contact at -4.9 pp, Size at -0.7 pp, and Text Layout at -0.7 pp (Wang et al., 4 Sep 2025). The authors explicitly suggest that for certain simpler concepts, rewriting may occasionally hurt rather than help.
The most important caution is methodological. The paper does not include an explicit ablation that replaces AlignEvaluator with CLIP, ImageReward, HPS, or no reward model while holding everything else fixed. It also does not separately evaluate the intrinsic accuracy of AlignEvaluator as a reward model through correlation with human judgments, AUC on preference prediction, or calibration statistics (Wang et al., 4 Sep 2025). Therefore, while the reported system-level gains support the value of fine-grained evaluator-driven reward shaping, they do not isolate AlignEvaluator’s standalone performance.
6. Relation to adjacent evaluator research
AlignEvaluator belongs to a broader family of evaluator systems, but its role is distinct. In text-to-image evaluation, "EvalAlign" is a human-aligned evaluator built by supervised fine-tuning a multimodal LLM to score image faithfulness and text-image alignment through structured question answering (Tan et al., 2024). AlignEvaluator differs in function: it is a reward model embedded inside a reinforcement-learning loop for prompt rewriting, rather than a benchmark metric for ranking image generators.
In evaluator research for LLMs, "AlignEval" evaluates an LLM’s alignment with human preferences without directly evaluating the model’s own generated answers, instead testing the model in the judge role on fixed pairwise comparisons (Liu et al., 25 Nov 2025). That is conceptually related at the level of evaluator design, but it addresses LLM alignment benchmarking rather than text-to-image prompt-image fidelity. "ARJudge" likewise addresses evaluator robustness by generating adaptive evaluation criteria and combining text-based and code-driven analyses for response comparison (Xu et al., 26 Feb 2025). These systems emphasize adaptive criteria and evaluation-process alignment, whereas AlignEvaluator emphasizes fine-grained T2I-specific reward shaping.
A plausible broader interpretation is that AlignEvaluator sits at the intersection of reward modeling and evaluator specialization: unlike static benchmarks, it is operationalized inside policy optimization; unlike generic multimodal judges, it is anchored to a fixed taxonomy of T2I failure modes. This also clarifies a frequent source of confusion: despite the similarity of names, AlignEvaluator is not the same object as AlignEval (Liu et al., 25 Nov 2025) or EvalAlign (Tan et al., 2024).
7. Limitations, caveats, and misconceptions
The paper identifies several explicit and implied limitations of AlignEvaluator (Wang et al., 4 Sep 2025). First, the reward model is bounded by the 24-key-point taxonomy. This gives it structure and interpretability, but any important failure mode outside those 24 dimensions may receive no appropriate reward signal. Second, because the evaluator ultimately emits a single scalar reward after internally considering 24 dimensions, some subtle categories may be underweighted; the weighting scheme is undisclosed. Third, the observed regressions in categories such as Interaction w/o Contact and Text Layout indicate that reward-guided rewriting can still create over-specification or ambiguity.
Further limitations follow from the paper’s omissions. Because it does not disclose detailed architecture, annotation, or calibration information, it is impossible from the paper alone to assess robustness to distribution shift, multilingual edge cases, or reward hacking (Wang et al., 4 Sep 2025). The paper also does not explicitly discuss reward hacking, though any reinforcement-learning setup with a learned reward model raises that possibility. Nor does it report inter-annotator agreement, evaluator correlation with human judgments, or benchmark reliability statistics.
Several misconceptions can therefore be addressed directly. AlignEvaluator is not an aesthetic judge; it is not a text-only prompt similarity model; it is not a fully specified standalone benchmark; and it is not clearly described as an inference-time selector. Its central contribution is narrower and more technical: it converts the qualitative notion of prompt fidelity into a structured scalar reward for optimizing a Chain-of-Thought prompt rewriter (Wang et al., 4 Sep 2025). In that sense, AlignEvaluator is less a general-purpose evaluator than the operational core of PromptEnhancer’s alignment mechanism.