SelfReVision in AI: Iterative Self-Revision

• SelfReVision is a self-improvement paradigm where models iteratively revise their internal reasoning and outputs to overcome sparse outcome supervision.
• It employs techniques like selective rewriting, dual-role architectures, and token-level self-distillation to optimize performance and correct errors.
• Empirical findings show improvements in accuracy, reasoning efficiency, and calibration across textual, multimodal, and embodied AI applications.

SelfReVision denotes a family of self-improvement mechanisms in which a model revises its own outputs, internal reasoning, evidence selection, or action plans and then reuses those revisions as supervision, control, or context for subsequent computation. In recent work, the revised object has ranged from chain-of-thought traces and correction prompts to video segments, scene layouts, and robot plans. The unifying premise is that sparse outcome supervision—such as final correctness, pass/fail verification, or one-shot quality signals—does not adequately constrain internal computation, so a model must be taught to inspect and restructure its own intermediate process rather than only optimize its terminal output (Yao et al., 20 Nov 2025, He et al., 13 Apr 2026, Lee et al., 20 Feb 2025, Li et al., 17 Nov 2025).

1. Conceptual scope and research lineage

In large reasoning models, SelfReVision is motivated by the claim that one-sided outcome-correctness rewards leave internal reasoning weakly supervised. The reported failure modes are over-thinking, under-thinking, redundant-thinking, and disordered-thinking: reasoning may be too long, too compressed on crucial steps, repetitious, or incoherent in flow. SelfReVision addresses this by turning the model’s own “thinking trace” into a revisable object and by learning from that revised trace inside training or inference loops rather than treating reasoning as an opaque by-product (Yao et al., 20 Nov 2025).

An earlier articulation of the broader idea appears in SELF, which separated meta-skill learning for self-feedback and self-refinement from later iterative self-evolution on unlabeled instructions. Parallel formulations specialized the paradigm in different ways: ReVISE modeled self-verification as a binary control decision between $[\mathtt{eos}]$ and $[\mathtt{refine}]$, while SD-Zero trained a single model to act as both Generator and Reviser, using binary outcome rewards to produce dense token-level self-supervision (Lu et al., 2023, Lee et al., 20 Feb 2025, He et al., 13 Apr 2026).

Across these systems, SelfReVision is not equivalent to self-scoring. Several papers explicitly distinguish it from self-rewarding LRMs that output only scalar judgments, from self-consistency methods that merely select among multiple traces, and from RLAIF or reward-model pipelines that depend on external preference models or human-labeled comparisons. In the SelfReVision formulation, the model often generates revised content, revised evidence, or revised control states, and those artifacts become the learning substrate (Yao et al., 20 Nov 2025, He et al., 13 Apr 2026).

2. Textual self-revision in reasoning and alignment

In the large-reasoning-model setting, SelfReVision was formalized as selective self-rewriting inside an online GRPO loop. For each query $q$, the method first samples $G/2$ rollouts. If all of those answers are correct, the query is treated as “simple,” and only then are the remaining half of the rollouts rewritten with a neutral editor prompt: “You are a skilled editor… rewrite the thinking passage to be more organized, coherent, and accurate, preserving core ideas; produce only the rewritten passage.” The rewritten thought is concatenated back with the original query, and the model continues generation to produce an answer. If the first half is not all correct, the second half remains vanilla generation, preserving exploration on harder queries and retaining the correctness-centric GRPO signal (Yao et al., 20 Nov 2025).

SD-Zero moved the same general idea into a two-role architecture within a single model. A Generator first samples an initial response $y_0 \sim p_\theta(\cdot \mid x)$, a binary verifier computes $r \in \{0,1\}$, and a Reviser conditions on $(x, y_0, r)$ to produce an improved response. Phase 1 trains outcome-conditioned self-revision with two losses, $L_{\text{revision}}$ and $L_{\text{generation}}$, and Phase 2 performs on-policy self-distillation from the reviser’s conditional next-token distributions back into the generator. The paper emphasizes two mechanisms: token-level self-localization, where KL mass concentrates on a small set of erroneous tokens for incorrect responses, and iterative self-evolution via periodic teacher synchronization (He et al., 13 Apr 2026).

ReVISE and REVES both target the mismatch between single-shot training and multi-step correction at test time, but they do so differently. ReVISE builds an intrinsic verifier by training the model to emit either $[\mathtt{eos}]$ or $[\mathtt{refine}]$0 after an initial trajectory, then conditions a revised second attempt on the refine token and the previous answer. REVES instead treats sequential revision as a state-distribution problem: successful recovery trajectories are mined for intermediate near-misses, and each near-miss is converted into a revision prompt and a verification prompt, so that policy improvement can focus on local correction and stopping behavior rather than only on terminal trajectory return (Lee et al., 20 Feb 2025, Liu et al., 17 Jun 2026).

SELF and Re5 extended the paradigm toward general instruction following and data-centric alignment. SELF trains self-feedback and self-refinement as meta-skills, then iteratively fine-tunes on self-curated refined outputs from unlabeled prompts. Re5 decomposes instructions into task and constraint components, applies a binary structural gate, performs constraint-specific evaluations for Format, Numeric, Length, and Content constraints, and then executes selective revision rather than wholesale regeneration. The resulting improved outputs are harvested for alignment tuning, including DPO on successful cases (Lu et al., 2023, Park, 8 Jul 2025).

3. Objectives, rewards, and control mechanisms

A distinctive feature of SelfReVision systems is that they convert revision into an explicit optimization object. In the selective self-rewriting framework, the shaped reward is

$[\mathtt{refine}]$1

where $[\mathtt{refine}]$2 denotes group-level correctness on the first half of the rollouts. GRPO-style clipped optimization is then applied with second-half samples drawn according to the selective rewriting policy, rewritten-group advantages are divided by $[\mathtt{refine}]$3, KL regularization is set to $[\mathtt{refine}]$4, and rewriting and vanilla generation are compiled into one joint inference batch so that the added walltime is only roughly $[\mathtt{refine}]$5 compared to vanilla GRPO (Yao et al., 20 Nov 2025).

SD-Zero replaces scalar-only supervision with token-level distillation. Its Phase 2 objective is

$[\mathtt{refine}]$6

implemented with top-$[\mathtt{refine}]$7 logit distillation where $[\mathtt{refine}]$8 and teacher distributions are StopGrad. Because the teacher is conditioned on the student’s actual attempt and its reward, the supervision is on-policy and error-specific rather than a generic imitation target. Regular teacher synchronization after the first epoch adds at least $[\mathtt{refine}]$9 gains on OpenR1-Math before plateauing (He et al., 13 Apr 2026).

ReVISE makes the verifier itself part of the generative policy. Its decision rule compares the model’s probabilities for $q$0 and $q$1, and confidence-aware decoding uses $q$2 as a weight in majority voting. REVES, by contrast, formalizes sequential correction through the hazard decomposition

$q$3

which reframes multi-turn improvement as local recovery probability at visited revision states. Its practical training objective is a decoupled combination,

$q$4

where revision prompts train answer transformation and verification prompts train correctness judgment and self-stopping calibration (Lee et al., 20 Feb 2025, Liu et al., 17 Jun 2026).

4. Multimodal, video, and embodied instantiations

In vision-language reasoning, SelfReVision has been implemented as explicit reflective continuation inside RL. VL-Rethinker augments GRPO with Selective Sample Replay to mitigate vanishing advantages and adds Forced Rethinking, in which a rethinking trigger is appended after an initial segment $q$5 so that the model continues with a reflective segment $q$6. The full output has the structure $q$7, and an auxiliary negative log-likelihood term is applied on the reflection segment when the final answer is correct. The goal is not merely longer output, but selective self-verification, self-correction, and self-questioning in multimodal reasoning (Wang et al., 10 Apr 2025).

Long-video work introduced a stronger requirement: reflection must revisit visual evidence, not only text. SelfReS, a closely related self-reflective sampler, uses the last token of the user query as a reflection token and computes Self-Reflective Attention,

$q$8

to select non-linear spatiotemporal fragments without additional training. REVISOR generalizes the idea into a two-stage introspective loop: an initial pass proposes both a reasoning trace $q$9 and a temporal segment $G/2$0, dense resampling is then performed only inside $G/2$1, and a second pass refines the answer using the reviewed frames. Its Dual Attribution Decoupled Reward separates final-answer correctness from Causal Segment Sufficiency Reward, explicitly aligning reasoning with the retrieved evidence (Pereira et al., 26 Mar 2025, Li et al., 17 Nov 2025).

In video editing, ReViSE turns an internal VLM into a differentiable critic. A DiT-based generator produces an edited video, a cleaner latent estimate $G/2$2 is decoded for evaluation, and the internal VLM outputs a strict Yes/No judgment over Edit Accuracy, Preservation Consistency, Generation Naturalness, and Generation Realism. The resulting logit-based reasoning loss is combined with flow matching in Unified Semantic Optimization. In 3D scene synthesis, SceneReVis realizes SelfReVision as a diagnose-and-act loop over rendered views, scene graphs, and interaction history, with primitive actions $G/2$3 and a hybrid reward that values physical plausibility, semantic progress, and final scene quality (Liu et al., 10 Dec 2025, Zhao et al., 10 Feb 2026).

Embodied procedural planning uses a lighter-weight but structurally similar pattern. In the VLM planning framework named SelfReVision, a model iteratively executes Critique–Revise–Verify over image-grounded plans, retains a revised plan only when self-verification answers “yes,” and then uses the improved plan both at inference time and as self-distilled SFT data. The revised plans are characterized by added low-level details such as preconditions, object-specific constraints, spatial relations, and failure contingencies, rather than generic textual polishing (Park et al., 11 Jul 2025).

5. Empirical behavior and evaluation

Across domains, empirical claims are usually framed in terms of process-sensitive metrics: reasoning length, internal quality scores, token efficiency, grounding quality, physical plausibility, or downstream executability.

System	Evaluation setting	Reported effect
SelfReVision (Yao et al., 20 Nov 2025)	Qwen3-8B	accuracy $G/2$4; reasoning length $G/2$5 tokens; judge $G/2$6
SD-Zero (He et al., 13 Apr 2026)	Qwen3-4B-Instruct avg@8	$G/2$7; pass@8 $G/2$8 vs Phase 1 $G/2$9 and RFT $y_0 \sim p_\theta(\cdot \mid x)$0
Re5 (Park, 8 Jul 2025)	DPO on $y_0 \sim p_\theta(\cdot \mid x)$1 improved cases	IFeval $y_0 \sim p_\theta(\cdot \mid x)$2; Multi-IF $y_0 \sim p_\theta(\cdot \mid x)$3; OQA-1 $y_0 \sim p_\theta(\cdot \mid x)$4 win-rate
REVISOR (Li et al., 17 Nov 2025)	Long-form video benchmarks	VideoMME $y_0 \sim p_\theta(\cdot \mid x)$5 vs $y_0 \sim p_\theta(\cdot \mid x)$6; MLVU $y_0 \sim p_\theta(\cdot \mid x)$7 vs $y_0 \sim p_\theta(\cdot \mid x)$8
SelfReVision for VLM planning (Park et al., 11 Jul 2025)	Places / Simulation	average Overall win rate $y_0 \sim p_\theta(\cdot \mid x)$9 / $r \in \{0,1\}$0; $r \in \{0,1\}$1B+ models beat GPT-4o by $r \in \{0,1\}$2 win rate
ReViSE video editing (Liu et al., 10 Dec 2025)	RVE-Bench reasoning-informed video editing	$r \in \{0,1\}$3 improvement of the Overall score over state-of-the-art methods

A plausible implication is that the strongest gains appear when revision improves the structure of computation rather than merely adding more samples or longer traces. Several ablations reinforce that interpretation: selective rewriting outperforms random rewriting in LRMs, verification improves calibration in sequential revision, and critique is essential in robot-friendly VLM planning.

6. Limitations, misconceptions, and contested points

A recurrent misconception is that self-revision automatically yields monotonic iterative improvement. The design study “Revision Matters” provides a direct counterexample. In pure self-revision, the model’s own output at round $r \in \{0,1\}$4 becomes the revision input for round $r \in \{0,1\}$5, with no learned judge or adaptive stopping criterion, and the result is an “echo chamber.” For the Direct $r \in \{0,1\}$6 model, duplication statistics rose from $r \in \{0,1\}$7 and RougeL $r \in \{0,1\}$8 at round 2 to $r \in \{0,1\}$9 and RougeL $(x, y_0, r)$0 at round 3; even higher decoding temperature improved round-1 FID but did not produce iterative improvement. By contrast, early human guidance was decisive: Multi-revision with multiple human edits reached FID $(x, y_0, r)$1, close to expert performance of approximately $(x, y_0, r)$2 (Li et al., 2024).

Another concern is self-preference bias: a model may resist valid corrections to its own drafts. A four-model test on verifiable instruction-following revision did not find such an effect under genuine authorship. On verified-good fixes, pooled rejection rates were $(x, y_0, r)$3 for authors and $(x, y_0, r)$4 for fresh models, with gap $(x, y_0, r)$5 percentage points and $(x, y_0, r)$6 CI $(x, y_0, r)$7. Moreover, $(x, y_0, r)$8 of author self-rejections were classified as stricter-than-checker flaw-catching rather than preference. The study therefore supports the weaker claim that self-preference is weak or absent in this narrowly verifiable setting, while noting that effects smaller than approximately $(x, y_0, r)$9 pp cannot be excluded at the available sample size (Guey et al., 18 Jun 2026).

Broader limitations recur across implementations. Selective rewriting may amplify preferred stylistic patterns on easy problems, does not rewrite hard cases, and can shorten useful content even without explicit length-minimization prompts. In token-level self-distillation, noisy or misaligned binary rewards can induce misleading teacher distributions, and teacher synchronization can destabilize if performed too frequently or too early. In long-video reflection, text-only rethinking is insufficient, segment mis-localization remains possible, and over-emphasis on evidence sufficiency can hurt end-task QA. In reason-informed video editing, overly large self-reflection weight $L_{\text{revision}}$0 can destabilize training or cause over-editing (Yao et al., 20 Nov 2025, He et al., 13 Apr 2026, Li et al., 17 Nov 2025, Liu et al., 10 Dec 2025).

Taken together, these results suggest that SelfReVision is most reliable when revision is tightly scoped, gated by verifiable feedback, and constrained to preserve already-correct structure. Under those conditions, it functions less as generic introspection than as a concrete mechanism for converting sparse success signals into process-level supervision.