CoT-Editing Consistency Reward

• CoT-Editing Consistency Reward is a framework that employs chain-of-thought reasoning to decompose visual and textual cues into actionable checklists and principles.
• It integrates adaptive planning and iterative correction steps to preserve key attributes like identity, style, and instruction fidelity in multimodal generation tasks.
• Empirical evaluations demonstrate improved consistency scores and interpretability, highlighting its efficacy in balancing preservation constraints with targeted modifications.

Searching arXiv for the cited papers and related context. arXiv search query: (Ye et al., 22 Dec 2025) CoT-Editing Consistency Reward denotes a reward-design pattern in which chain-of-thought-style reasoning is used to make image generation or image editing explicitly check consistency constraints rather than optimize only a holistic scalar notion of quality. In the formulation of "Visual-Aware CoT: Achieving High-Fidelity Visual Consistency in Unified Models," the reward combines a checklist-conditioned visual consistency term with a text-alignment term for multimodal generation with visual references (Ye et al., 22 Dec 2025). In "Leveraging Verifier-Based Reinforcement Learning in Image Editing," the analogous move is from a simple scorer to a reasoning verifier that decomposes an editing instruction into principles, verifies them point-wise, and aggregates those checks into a downstream reinforcement-learning reward for editing models (Guo et al., 30 Apr 2026).

1. Conceptual basis

The central motivation is that standard chain-of-thought and standard reward modeling are often too coarse for editing-style tasks. In unified multimodal generation, existing CoT methods are described as focusing on text alignment with the user prompt, which improves textual alignment but does not explicitly preserve visual attributes such as identity, object properties, and style across multiple references (Ye et al., 22 Dec 2025). In image editing, existing reward models are described as usually giving overall scores without detailed checks, even though editing must simultaneously judge instruction fidelity, preservation of untargeted content, local versus global consistency, and visual quality (Guo et al., 30 Apr 2026).

These two lines of work target the same structural deficiency from different angles. One paper begins from multimodal generation and asks whether the output preserves the visual evidence contained in the reference images; the other begins from image-editing reward modeling and asks whether evaluation should be decomposed into verifiable principles rather than collapsed into a single blunt score. This suggests that “CoT-editing consistency reward” is best understood not as a single canonical objective, but as a family of reward constructions in which reasoning traces are trained to verify preservation constraints, requested modifications, and output fidelity.

A common misconception is that such a reward is equivalent to generic prompt following or generic image quality scoring. The cited work argues against that equivalence. In the multimodal case, the stronger question is not merely whether the generated image matches the text prompt, but whether it also preserves the important visual evidence contained in the reference images. In the editing case, the final score is explicitly not treated as a simple average of principle scores; the model must also assess edit magnitude, usability, and whether changes outside the instruction are unreasonable (Ye et al., 22 Dec 2025, Guo et al., 30 Apr 2026).

2. Checklist-conditioned visual consistency in unified multimodal generation

In VACoT, the problem setting is multi-modal generation with visual references, especially multi-reference image generation and editing. The model receives a text prompt $T$ and visual context $V=\{v_1,v_2,\ldots,v_n\}$ consisting of multiple reference images, and must generate an output image $Y$ via a $\mathcal{T}$-step denoising or reverse-trajectory process $\{x_{\mathcal{T}},x_{\mathcal{T}-1},\ldots,x_0=Y\}$ (Ye et al., 22 Dec 2025).

The method is built around two components.

First, Adaptive Visual Planning generates a structured visual checklist $Z_{\text{plan}}$. Each item identifies a visual aspect that should be checked against the reference images. The checking types are categorized as identity, style, and attribute. Each checklist item is defined as

$z_i=\{check\_type, source, target\}.$

Here, source and target specify which element from the input visual context should be preserved in which part of the generated output. The checklist therefore converts a vague requirement of “visual consistency” into structured reasoning targets (Ye et al., 22 Dec 2025).

Second, Iterative Visual Correction evaluates the current generated image against that checklist and the visual context:

$$Z_{\text{eval}} = f_{\text{evaluate}}(Y_{\text{current}}, Z_{\text{plan}}, V, T),$$

followed by correction:

$$Y_{\text{corrected}} = f_{\text{edit}}(T, V, Z_{\text{plan}}, Y_{\text{current}}, Z_{\text{eval}}).$$

The resulting loop is plan $\rightarrow$ generate $V=\{v_1,v_2,\ldots,v_n\}$0 evaluate $V=\{v_1,v_2,\ldots,v_n\}$1 correct $V=\{v_1,v_2,\ldots,v_n\}$2 repeat, terminating either when the evaluation indicates satisfaction or when the maximum number of iterations $V=\{v_1,v_2,\ldots,v_n\}$3 is reached (Ye et al., 22 Dec 2025).

The base model is BAGEL, described as a unified decoder-only Transformer with Mixture-of-Transformer-Experts that shares multimodal token sequences through self-attention while using separate experts for understanding and generation. Because it can interleave text and image tokens during inference, the multimodal CoT loop is feasible inside a single architecture (Ye et al., 22 Dec 2025).

3. Reward decomposition and optimization in VACoT

VACoT operationalizes visual context consistency through a reward decomposition rather than through a standalone abstract metric. The core formulation is

$V=\{v_1,v_2,\ldots,v_n\}$4

Here, $V=\{v_1,v_2,\ldots,v_n\}$5 measures consistency with the visual context and checklist, while $V=\{v_1,v_2,\ldots,v_n\}$6 measures textual alignment with the prompt (Ye et al., 22 Dec 2025).

The visual reward is checklist-aware and type-specific. For identity preservation, GroundingDINO is used to locate the target object in the reference and generated images, and DINO similarity is computed between the corresponding cropped regions. For style consistency, the method uses CSD-Score. For attribute consistency, the paper states that checklist items are dynamically evaluated by type-specific visual similarity metrics, although the main text explicitly names identity and style metrics (Ye et al., 22 Dec 2025).

The text reward $V=\{v_1,v_2,\ldots,v_n\}$7 is computed using CLIP score. Under flow-GRPO, the group-relative advantage for group size $V=\{v_1,v_2,\ldots,v_n\}$8 is

$V=\{v_1,v_2,\ldots,v_n\}$9

The policy is then optimized with a clipped objective and a KL penalty to a reference policy, with ratio

$Y$0

The important point is that the reward is not treated as a generic image-quality scalar; it is a checklist-conditioned visual consistency reward integrated into online RL (Ye et al., 22 Dec 2025).

The supplementary material reports that on the training dataset, in 78.3\% of samples the ground truth image gets a higher object similarity score than the sub-optimal generation. A plausible implication is that the identity-oriented component of the reward is reasonably aligned with the desired notion of reference preservation, although the paper does not elevate this observation into a universal guarantee (Ye et al., 22 Dec 2025).

4. Principle-wise verifier rewards for image editing

Edit-R1 generalizes the same consistency-oriented logic to image editing by replacing a holistic reward scorer with a reasoning verifier. Its reward model, Edit-RRM, decomposes the editing instruction into a principle set

$Y$1

with principles organized into three categories: Keep, Follow, and Quality (Guo et al., 30 Apr 2026).

Each training example is represented as

$Y$2

where $Y$3 is the source image, $Y$4 is the editing instruction, $Y$5 is a generated candidate, and $Y$6 is the decomposed principle set. The decomposition prompt asks for around 10 evaluation points, usually split roughly as 3–4 “Instruction Following” points, 3–4 “Feature Preservation” points, and 2–3 “Image Quality” points (Guo et al., 30 Apr 2026).

At inference time, the reasoning verifier compares original and edited images side-by-side, scores each principle as 0 or 1, reasons holistically about overall usability, considers unmentioned but important changes, and penalizes structural artifacts or unreasonable changes. The scalar reward is extracted from the textual output through

$Y$7

where $Y$8 is the reward model, $Y$9 is the context, and $\mathcal{T}$0 parses the final score from the generated text (Guo et al., 30 Apr 2026).

The paper explicitly states that the final score should not be a simple average of point scores. Instead, the verifier must also assess edit magnitude, usability, and whether changes outside the instruction are unreasonable. That requirement sharply distinguishes the approach from naïve principle counting. It is still a scalar reward in the end, but one produced by a reasoning process that preserves the internal structure of editing requirements (Guo et al., 30 Apr 2026).

5. Training pipelines: supervised reasoning traces and reinforcement learning

The two systems share a two-stage pattern: supervised learning first establishes the reasoning behavior, and reinforcement learning then sharpens it with consistency-oriented rewards.

In VACoT, supervised finetuning uses a planning dataset

$\mathcal{T}$1

and a correction dataset

$\mathcal{T}$2

The planning data teach checklist generation, while the correction data teach self-reflection and self-refinement, including “perfect result” cases in which no further editing is needed. Reinforcement learning then uses flow-GRPO with the visual-consistency reward just described (Ye et al., 22 Dec 2025).

In Edit-R1, the cold-start SFT stage uses about 200K samples from a public image-editing benchmark: 100K random samples and 100K hard samples selected by GPT-4o. Multiple image-editing models produce about 2 million quadruples, and a pool of VLMs generates multiple “Think + Score” trajectories for each quadruple. SeedVLM-1.5 then re-evaluates those candidate traces and the trace with the highest verification accuracy is selected as the supervised target for SFT (Guo et al., 30 Apr 2026).

After cold-start SFT, Edit-R1 applies Group Contrastive Preference Optimization (GCPO) to the reward model itself using about 10,000 human-annotated preference pairs. For a preference pair $\mathcal{T}$3 and $\mathcal{T}$4 sampled reasoning-score trajectories per image, the win/loss ratios are

$\mathcal{T}$5

The centered group rewards are

$\mathcal{T}$6

The downstream editor is then trained with GRPO using scores from the learned verifier, with group size $\mathcal{T}$7 and KL coefficient $\mathcal{T}$8 (Guo et al., 30 Apr 2026).

System	Decomposition unit	Reward target
VACoT	Structured visual checklist	Visual-context consistency plus text alignment
Edit-R1 / Edit-RRM	Principle set in Keep / Follow / Quality	Verifier score aggregated from principle-wise checks

This comparison shows a shared architectural logic. One system decomposes what must stay visually stable across references; the other decomposes what must be kept, followed, and preserved in editing. This suggests a broader consistency-reward template in which CoT is not merely explanatory text, but an operational scaffold for reward construction.

6. Empirical behavior, interpretability, and limitations

VACoT reports strong evidence that the consistency-oriented reward is not interchangeable with text-only CoT. On OmniContext, the main table reports 8.26 average for the method, compared with 7.89 for Uni-CoT, 6.85 for UiG, and 5.55 for BAGEL. In the ablation on training stages, the scores are 6.02 for the BAGEL baseline, 7.89 for BAGEL + VACoT mechanism, 8.06 for Ours w/o SFT, 8.13 for Ours w/o GRPO, and 8.44 for the full method. In the supplementary reward ablation, ObjSimScore + CLIPScore achieves 8.44, compared with 8.08 for ObjSimScore alone and 7.49 for ObjSimScore + PickScore + CLIPScore. The paper also reports that most correctable cases are fixed within about 2–3 iterations, and that additional iterations can hurt because of over-correction (Ye et al., 22 Dec 2025).

Edit-R1 reports evidence at both reward-model and downstream-editing levels. In the internal benchmark, Qwen-3B after full SFT+GCPO reaches 72.0\%, and Qwen-7B reaches 82.2\%. For Qwen-7B, “Think” only gives 68.9\%, while “Think + Verify” gives 75.4\%. On EditRewardBench, the paper reports 65.9\% for EditScore-7B, 72.7\% for EditScore-7B + inference scaling, 73.3\% for the RRM with SFT only, and 78.2\% for the RRM with SFT + GCPO. The final 7B RL-RRM surpasses Seed-1.5-VL at 79.3\% and Seed-1.6-VL at 77.2\%. When used to train editors, FLUX.Kontext improves from overall score 5.77 \rightarrow 6.24 and semantic consistency 6.27 \rightarrow 6.86; for Qwen-Edit on the Motion Change category, the score improves from 4.01 \rightarrow 4.62, a 15.2\% relative improvement; and FLUX.Kontext w. RL-RRM (7B) gets GSB +23.2 in human evaluation (Guo et al., 30 Apr 2026).

Interpretability is a major stated benefit in both lines of work. VACoT’s checklist and evaluation loop expose which identity, style, or attribute constraints are unsatisfied. Edit-RRM’s reasoning trace reveals which principles were met, which were not, and why a final score was assigned. This makes failure analysis more localized than in holistic scalar scorers (Ye et al., 22 Dec 2025, Guo et al., 30 Apr 2026).

The limitations are similarly explicit. VACoT is designed to correct identity distortion, recover missing identity features, remove unreasonable elements, and support style transfer in complex multi-reference generation, but the paper also notes that too many correction iterations can degrade results through over-correction (Ye et al., 22 Dec 2025). Edit-RRM, even after cold-start SFT, can hallucinate, misjudge edit magnitude, and incorrectly mark a small movement as satisfying a positional instruction. The system also depends on VLM-generated decomposition and verification, and remains fallible on subtle geometric or magnitude-based constraints (Guo et al., 30 Apr 2026).

Taken together, these works define CoT-editing consistency reward as a shift from prompt-only or scorer-only supervision toward decomposition, verification, and preservation-aware reinforcement learning. In one case, the reward is reference-conditioned and type-aware over identity, style, and attributes. In the other, it is principle-driven over Keep, Follow, and Quality. The common result is a reward signal that is meant to evaluate not only whether an image is plausible or instruction-aligned, but whether the relevant evidence has been kept intact while the requested transformation is carried out (Ye et al., 22 Dec 2025, Guo et al., 30 Apr 2026).