Conditional Step-level Self-Refinement

• Conditional step-level self-refinement is a method where a model or auxiliary verifier assesses intermediate reasoning steps and revises them only when local evidence indicates errors.
• The approach employs diverse units such as chain-of-thought steps, thought–action pairs, and token events, triggering revisions based on confidence thresholds or anomaly detection.
• It enhances accuracy and efficiency in AI applications by localizing corrections rather than reworking entire outputs, thus preserving correct reasoning.

Conditional step-level self-refinement denotes a class of methods in which a model, or an auxiliary verifier attached to it, evaluates intermediate reasoning units and revises them only when local evidence indicates that revision is warranted. The “step” may be a reasoning step in chain-of-thought, a thought–action pair in an agent trajectory, a sub-question/sub-answer pair, a prompt atom, an action in an interactive environment, or even a token–depth event inside a transformer. The “conditional” aspect is equally central: unlike unconditional rewrite loops, these systems refine only when a trigger such as a low step score, failed entailment check, low confidence estimate, or abnormal internal-flow pattern is observed. As a research lineage, the topic connects early whole-output iterative refinement in “Self-Refine” (Madaan et al., 2023) to later explicitly step-wise systems for search agents, mathematical reasoning, test-time scaling, interactive agents, multimodal prompt refinement, and internal-state intervention (Zhang et al., 8 Feb 2026, Yan et al., 2024, Chang et al., 21 Jul 2025, Xiong et al., 2024, Shi et al., 13 Nov 2025, Jeong et al., 2 Feb 2026, Yang et al., 7 Apr 2026).

1. Conceptual foundations

Conditional step-level self-refinement arose from dissatisfaction with outcome-only supervision. In search agents, sparse trajectory-level rewards obscure where a reasoning process failed, because a correct final answer may still be preceded by wasted or logically poor steps, while an incorrect answer may result from a single early mistake (Zhang et al., 8 Feb 2026). In mathematical reasoning, post-hoc correction can miss local errors that propagate through an otherwise coherent derivation, motivating mechanisms that recognize and repair faulty intermediate steps during or immediately after they are produced (Yan et al., 2024). In interactive agents, trajectory-level success labels similarly fail to distinguish useful local actions from detours, blind exploration, or semantically wrong operations (Xiong et al., 2024).

A general formal template was already implicit in Self-Refine. For input $x$, a model $\mathcal{M}$ produces an initial output $y_0$, feedback $fb_t$, and a refinement $y_{t+1}$ conditioned on the entire history: $$y_0 = \mathcal{M}(p_{\text{gen}} \| x), \qquad fb_t = \mathcal{M}(p_{\text{fb}} \| x \| y_t),$$

$$y_{t+1} = \mathcal{M}(p_{\text{ref}} \| x \| y_0 \| fb_0 \| \dots \| y_t \| fb_t).$$

The same paper explicitly notes that this scheme can be localized to substructures such as reasoning steps, with step-specific feedback and selective step replacement rather than full-output rewriting (Madaan et al., 2023). Later work operationalized that localization.

Across the literature, conditionality is implemented in several distinct but convergent ways. SRR-Judge refines a search step only when its rating falls below a threshold $\tau=4$ (Zhang et al., 8 Feb 2026). Verifier-guided hybrid test-time scaling triggers step reflection only when a process reward model score is below $0.9$, then stops when the step becomes confident enough or improvement stalls (Chang et al., 21 Jul 2025). SSR selects the least reliable Socratic step $t'=\arg\min_t c_t$ for targeted repair, and in its adaptive variants invokes the Socratic machinery only when a cheaper Self-Refine judge gives maximal confidence and therefore fails to expose an error (Shi et al., 13 Nov 2025). S$\mathcal{M}$0c-Math internalizes the keep-versus-revise decision as part of the LLM’s own generative policy, so that a model may continue normally or insert a correction segment inline, without any explicit external trigger at inference time (Yan et al., 2024).

2. Step granularity and representational choices

The central design variable is what counts as a “step.” Different research programs make different representational commitments, and the resulting mechanisms are not interchangeable.

System	Step unit	Refinement action
Self-Refine	Full current output $\mathcal{M}$1	Critique then rewrite output
SRR-Judge	Thought–action pair $\mathcal{M}$2	Replace with refined thought and action
S$\mathcal{M}$3c-Math	Intermediate CoT step	Insert reflection and corrected step inline
IPR	Agent action at prefix $\mathcal{M}$4	Prefer higher-reward suffix at that state
Verifier-guided TTS	CoT step delimited by \n\n	Critique and rewrite only that step
SSR	Socratic pair $\mathcal{M}$5	Repair lowest-confidence sub-answer
SCMAPR	Semantic atom or violated prompt constraint	Minimal prompt revision until all atoms are entailed
Internal Flow Signatures	Token–depth event $\mathcal{M}$6	Clamp abnormal transported step at one block

The literature therefore spans at least four levels of granularity. The first is textual whole-output refinement, exemplified by Self-Refine, which is step-wise in the iterative sense but not localized inside the reasoning trace (Madaan et al., 2023). The second is explicit reasoning-step refinement, as in verifier-guided TTS and SSR, where the units are human-readable intermediate deductions (Chang et al., 21 Jul 2025, Shi et al., 13 Nov 2025). The third is action-level refinement in embodied, web, or search agents, where a step combines reasoning and externally executed behavior (Zhang et al., 8 Feb 2026, Xiong et al., 2024). The fourth is latent computational refinement, where the unit is not a textual step at all but a block-level internal update inside the transformer (Jeong et al., 2 Feb 2026).

These choices matter because they determine both observability and controllability. Socratic decomposition in SSR assumes that a chain-of-thought can be represented as a sequence $\mathcal{M}$7, so verification reduces to re-solving sub-questions under controlled context (Shi et al., 13 Nov 2025). SRR-Judge instead models a search trajectory

$\mathcal{M}$8

and evaluates the local pair $\mathcal{M}$9 given the history $y_0$0 (Zhang et al., 8 Feb 2026). SCMAPR atomizes a prompt into characters, objects, actions, locations, and scenery, turning prompt refinement into constraint satisfaction over atoms rather than over free-form prose (Yang et al., 7 Apr 2026). Internal Flow Signatures treat each token’s hidden-state evolution $y_0$1 across transformer depth as a trajectory to be audited geometrically (Jeong et al., 2 Feb 2026).

A plausible implication is that “conditional step-level self-refinement” is better understood as a family of control architectures than as a single algorithm. What unifies the family is not a specific representation, but the sequence: localize an intermediate unit, assess it with context, and intervene only when the assessment indicates that intervention is beneficial.

3. Verification signals and conditional triggers

The step-level verifier is the core component in most systems. Its job is not merely to score quality globally, but to produce a signal sharp enough to decide whether a specific intermediate unit should be left untouched, edited locally, or replaced.

SRR-Judge is a generative judge fine-tuned from QwQ-32B. For each step $y_0$2, it implements

$y_0$3

where $y_0$4, $y_0$5 is a textual explanation, and $y_0$6 is a refined thought–action pair (Zhang et al., 8 Feb 2026). The rubric covers Clarity & Conciseness, Logical Structure, Query Appropriateness, and Coverage & Improvement Potential. In the modified ReAct loop, the base agent proposes $y_0$7 candidates, SRR-Judge rates them, and the system either keeps the best raw candidate if $y_0$8 or replaces it with the best refined candidate if all scores fall below $y_0$9. This is a canonical thresholded trigger.

Verifier-guided hybrid TTS uses a separate PRM: $fb_t$0 where $fb_t$1 is the problem, $fb_t$2 is the previous context, and $fb_t$3 is the current reasoning step (Chang et al., 21 Jul 2025). The step is isolated by a “Pause-then-Continue” scheme with \n\n as delimiter. Reflection is triggered only when $fb_t$4, the refined step is accepted only if its PRM score improves on the original, and refinement stops after at most $fb_t$5 iterations, when the PRM score exceeds $fb_t$6, or when the score improvement over two consecutive rounds is less than $fb_t$7. The acceptance rule is therefore verifier-monotone rather than rewrite-always.

SSR replaces scalar step scoring with controlled local re-solving. After decomposing a CoT into $fb_t$8, it samples a reference set $fb_t$9 for each sub-question $y_{t+1}$0, then assigns a confidence $y_{t+1}$1 by prompting the model to compare $y_{t+1}$2 against $y_{t+1}$3 on a $y_{t+1}$4–$y_{t+1}$5 scale (Shi et al., 13 Nov 2025). The selected target is the least reliable step,

$y_{t+1}$6

and refinement takes the form of “Socratic feedback” injected into a new full-chain regeneration. The conditional trigger is thus rank-based rather than threshold-based.

SCMAPR uses structured semantic verification instead of step scoring. It extracts atoms $y_{t+1}$7 from the original prompt, matches them to chunks in the rewritten prompt via embeddings, and labels each atom–chunk pair as ET, MS, or CT using an entailment validator (Yang et al., 7 Apr 2026). Acceptance is hard-gated by

$y_{t+1}$8

Any missing or contradictory atom triggers a minimal revision loop. Although developed for text-to-video prompting, the mechanism is a direct instance of conditional local repair.

Internal Flow Signatures move the verifier inside the transformer. After bias-centered monitoring and construction of moving readout-aligned subspaces, the method extracts transported step lengths $y_{t+1}$9, turning angles $$y_0 = \mathcal{M}(p_{\text{gen}} \| x), \qquad fb_t = \mathcal{M}(p_{\text{fb}} \| x \| y_t),$$0, channel magnitudes, residual ratios, and subspace drift summaries (Jeong et al., 2 Feb 2026). A lightweight GRU validator predicts whether the final answer is hallucinated and localizes a culprit event $$y_0 = \mathcal{M}(p_{\text{gen}} \| x), \qquad fb_t = \mathcal{M}(p_{\text{fb}} \| x \| y_t),$$1. Refinement is then triggered only for high validator scores, and only at the identified token and block: the model rolls back to token $$y_0 = \mathcal{M}(p_{\text{gen}} \| x), \qquad fb_t = \mathcal{M}(p_{\text{fb}} \| x \| y_t),$$2 and clamps an abnormal transported step at block $$y_0 = \mathcal{M}(p_{\text{gen}} \| x), \qquad fb_t = \mathcal{M}(p_{\text{fb}} \| x \| y_t),$$3 while preserving the orthogonal residual. This is conditional step-level refinement at internal-computation granularity.

4. Training regimes: from prompting to internalized correction

A major axis of variation is whether refinement remains an inference-time prompt pattern or becomes a trained policy. The literature contains training-free prompting methods, supervised internalization of correction behavior, and reinforcement-style optimization using step-level signals.

Self-Refine is the clearest training-free baseline. A single LLM serves as generator, feedback provider, and refiner, with no supervised training data, additional training, or reinforcement learning (Madaan et al., 2023). The same paper shows that specific, actionable feedback matters substantially more than generic feedback or repeated sampling without explicit critique. This established the basic generate–critique–refine pattern but left open how to make refinement reliably local and conditional.

S$$y_0 = \mathcal{M}(p_{\text{gen}} \| x), \qquad fb_t = \mathcal{M}(p_{\text{fb}} \| x \| y_t),$$4c-Math internalizes correction into the model’s generative distribution. Starting from MetaMathQA, it constructs S$$y_0 = \mathcal{M}(p_{\text{gen}} \| x), \qquad fb_t = \mathcal{M}(p_{\text{fb}} \| x \| y_t),$$5c-MathQA by inserting wrong steps, reflection spans, improvement spans, and corrected steps into otherwise correct solutions, yielding $$y_0 = \mathcal{M}(p_{\text{gen}} \| x), \qquad fb_t = \mathcal{M}(p_{\text{fb}} \| x \| y_t),$$6k self-correction examples; combined with the original $$y_0 = \mathcal{M}(p_{\text{gen}} \| x), \qquad fb_t = \mathcal{M}(p_{\text{fb}} \| x \| y_t),$$7k MetaMathQA examples, the final supervised corpus is $$y_0 = \mathcal{M}(p_{\text{gen}} \| x), \qquad fb_t = \mathcal{M}(p_{\text{fb}} \| x \| y_t),$$8k examples (Yan et al., 2024). Training uses a masked SFT objective

$$y_0 = \mathcal{M}(p_{\text{gen}} \| x), \qquad fb_t = \mathcal{M}(p_{\text{fb}} \| x \| y_t),$$9

where tokens inside wrong-step spans have $$y_{t+1} = \mathcal{M}(p_{\text{ref}} \| x \| y_0 \| fb_0 \| \dots \| y_t \| fb_t).$$0. The model is not trained to reproduce the erroneous steps, only to continue from erroneous context with “indicator + reflection + improvement + corrected step.” At inference time, correction is spontaneous: the model may insert “Sorry, I made a mistake” and proceed with a repaired derivation in a single pass (Yan et al., 2024).

IPR operationalizes step-level refinement for interactive agents by estimating a Monte Carlo step reward

$$y_{t+1} = \mathcal{M}(p_{\text{ref}} \| x \| y_0 \| fb_0 \| \dots \| y_t \| fb_t).$$1

which approximates the expected final outcome after taking action $$y_{t+1} = \mathcal{M}(p_{\text{ref}} \| x \| y_0 \| fb_0 \| \dots \| y_t \| fb_t).$$2 at state $$y_{t+1} = \mathcal{M}(p_{\text{ref}} \| x \| y_0 \| fb_0 \| \dots \| y_t \| fb_t).$$3 and then following a fixed scorer policy (Xiong et al., 2024). Expert and agent actions are compared at the same prefix, producing contrastive step-level suffix pairs for a mixed objective

$$y_{t+1} = \mathcal{M}(p_{\text{ref}} \| x \| y_0 \| fb_0 \| \dots \| y_t \| fb_t).$$4

Here, refinement is not a textual rewrite loop but a policy update toward locally better continuations.

SRR-Judge combines a trained step judge with iterative rejection-sampling fine-tuning. DeepSeek-V3.1 is sampled five times per step under the SRR prompt, the majority vote provides the step rating, and trajectories are filtered using point-biserial correlation between average step ratings and final correctness (Zhang et al., 8 Feb 2026). The resulting “silver” data train the judge via SFT, and the policy is then optimized by rejection-sampling fine-tuning on trajectories that are both correct and high-rated at the step level. The explicit comparison against ORM shows that conditioning trajectory acceptance on step quality matters, not merely final correctness.

ThinkTwice applies online policy optimization to self-refinement, but at whole-solution rather than explicit per-step granularity. In each pair of training steps, the model is first optimized on direct reasoning and then on refining its own solutions to the same problems, using the same binary correctness reward in both phases (Jiao et al., 2 Apr 2026). The analysis reports an implicit rectify-then-fortify curriculum: early in training, refinement mainly fixes wrong answers; later, it increasingly preserves correct ones. Although not step-level in the strict sense, it demonstrates that conditional “change versus keep” behavior can emerge from shared outcome rewards alone.

GSR likewise operates at whole-solution granularity, but is important because it treats refinement as a learnable skill. A unified model generates $$y_{t+1} = \mathcal{M}(p_{\text{ref}} \| x \| y_0 \| fb_0 \| \dots \| y_t \| fb_t).$$5 candidate solutions in parallel and then synthesizes a new superior solution conditioned on the problem and the candidates; hybrid training jointly optimizes direct solving and refinement (Wang et al., 27 Aug 2025). The paper explicitly argues that the same pattern can be adapted to step-level refinement by replacing full-solution candidates with candidate next steps or partial chains. This suggests that step-level self-refinement can be viewed as a granularity shift rather than a fundamentally different learning problem.

5. Empirical performance across domains

The empirical record is unusually broad. Conditional step-level refinement has been studied on natural-language generation, mathematical reasoning, web and embodied agents, search agents, hallucination detection, and text-to-video prompt engineering.

In early whole-output refinement, Self-Refine reported that outputs generated with iterative feedback and refinement were preferred by humans and automatic metrics over one-step generation, improving by ~20% absolute on average across $$y_{t+1} = \mathcal{M}(p_{\text{ref}} \| x \| y_0 \| fb_0 \| \dots \| y_t \| fb_t).$$6 tasks (Madaan et al., 2023). The task-level gains are highly heterogeneous but large: for GPT-4, code optimization improved from $$y_{t+1} = \mathcal{M}(p_{\text{ref}} \| x \| y_0 \| fb_0 \| \dots \| y_t \| fb_t).$$7 programs optimized to $$y_{t+1} = \mathcal{M}(p_{\text{ref}} \| x \| y_0 \| fb_0 \| \dots \| y_t \| fb_t).$$8, dialogue preference from $$y_{t+1} = \mathcal{M}(p_{\text{ref}} \| x \| y_0 \| fb_0 \| \dots \| y_t \| fb_t).$$9 to $\tau=4$0, sentiment reversal from $\tau=4$1 to $\tau=4$2, and constrained generation coverage from $\tau=4$3 to $\tau=4$4 (Madaan et al., 2023). The same paper also found that most gains occur in the first $\tau=4$5–$\tau=4$6 iterations.

In search-integrated reasoning, SRR-Judge established that a specialized step judge can be both reliable and actionable. On about $\tau=4$7k trajectories, SRR-Judge achieved average-across-steps point-biserial correlation $\tau=4$8 and last-step correlation $\tau=4$9, surpassing even its teacher DeepSeek-V3.1 and Qwen3-235B-A22B (Zhang et al., 8 Feb 2026). At inference time, the rate-and-refine loop improved QwQ-32B on BrowseComp from $0.9$0 to $0.9$1, BrowseComp-ZH from $0.9$2 to $0.9$3, and Xbench-DeepSearch from $0.9$4 to $0.9$5; with iterative SRR-based RFT, QwQ-32B reached $0.9$6 on BrowseComp/BrowseComp-ZH/Xbench-DeepSearch at iteration 1 and $0.9$7 at iteration 2, corresponding to over a $0.9$8 percent average absolute pass@1 improvement across deep-search benchmarks (Zhang et al., 8 Feb 2026).

In mathematical reasoning, S$0.9$9c-Math showed that spontaneous in-trajectory correction improves both small and large models. For Meta-Llama-3-8B, GSM8K rose from $t'=\arg\min_t c_t$0 to $t'=\arg\min_t c_t$1 and MATH from $t'=\arg\min_t c_t$2 to $t'=\arg\min_t c_t$3; for DeepSeek-Math-Base-7B, GSM8K rose from $t'=\arg\min_t c_t$4 to $t'=\arg\min_t c_t$5 and MATH from $t'=\arg\min_t c_t$6 to $t'=\arg\min_t c_t$7; for Meta-Llama-3-70B, GSM8K rose from $t'=\arg\min_t c_t$8 to $t'=\arg\min_t c_t$9 and MATH from $\mathcal{M}$00 to $\mathcal{M}$01 (Yan et al., 2024). Ablations further showed that step-level correction supervision outperformed instance-level correction and MCTS-generated correction data.

Verifier-guided training-free test-time scaling pushed step-level refinement even further. On MATH500 with Qwen2.5-7B-Instruct, one round of step-level self-refinement reached $\mathcal{M}$02, versus $\mathcal{M}$03 for both the baseline and solution-level self-refinement; after $\mathcal{M}$04 rounds, step-level self-refinement reached $\mathcal{M}$05 while solution-level self-refinement remained near $\mathcal{M}$06–$\mathcal{M}$07 (Chang et al., 21 Jul 2025). Inside Hybrid TTS, the gains became much larger: on MATH500, Qwen2.5-3B improved from $\mathcal{M}$08 to $\mathcal{M}$09, Qwen2.5-7B from $\mathcal{M}$10 to $\mathcal{M}$11, and LLaMA3.1-8B from $\mathcal{M}$12 to $\mathcal{M}$13 under Best-of-16 Pass@16; on GPQA Diamond, Qwen2.5-3B improved from $\mathcal{M}$14 to $\mathcal{M}$15 (Chang et al., 21 Jul 2025).

SSR supplied a black-box step-level alternative based on Socratic decomposition. On GPT-5-mini, LR-Acc on MATH Level-5 improved from $\mathcal{M}$16 for CoT to $\mathcal{M}$17 for SSR-Plan; on AIME24, from $\mathcal{M}$18 to $\mathcal{M}$19 (Shi et al., 13 Nov 2025). On the Humanity’s Last Exam math subset, GPT-5 improved from $\mathcal{M}$20 with CoT to $\mathcal{M}$21 with SSR-Plan, while Self-Refine actually dropped to $\mathcal{M}$22 (Shi et al., 13 Nov 2025). This is important because it shows a regime in which coarse whole-solution refinement fails but conditional step-level refinement still helps.

Interactive-agent learning shows similar effects. On WebShop, InterCodeSQL, and ALFWorld, IPR achieved average reward $\mathcal{M}$23, outperforming ETO at $\mathcal{M}$24 and Step-PPO at $\mathcal{M}$25; the reported improvements over ETO were $\mathcal{M}$26 on WebShop, $\mathcal{M}$27 on InterCodeSQL, and around $\mathcal{M}$28 on average across ALFWorld seen and unseen splits (Xiong et al., 2024). The process metric “average reward per step” also improved consistently.

Whole-solution RL self-refinement remains competitive, but its empirical profile is distinct. ThinkTwice reported that on Qwen3-4B it outperformed GRPO on AIME by $\mathcal{M}$29 percentage points before refinement and by $\mathcal{M}$30 points after one self-refinement step, measured by pass@4 (Jiao et al., 2 Apr 2026). The result is notable because it isolates the benefit of joint training for initial reasoning and later refinement, even without explicit step-level supervision.

Outside pure reasoning benchmarks, the same pattern appears in prompt engineering and hallucination control. SCMAPR improved average score on VBench by up to $\mathcal{M}$31, on EvalCrafter by up to $\mathcal{M}$32, and on T2V-CompBench by up to $\mathcal{M}$33 over $\mathcal{M}$34 state-of-the-art baselines, using scenario routing, policy-conditioned rewriting, atom-level semantic verification, and conditional revision (Yang et al., 7 Apr 2026). Internal Flow Signatures, operating inside the transformer, reduced hallucination ratios in QA by up to about $\mathcal{M}$35 relative; for example, LLaMA3 dropped from $\mathcal{M}$36 to $\mathcal{M}$37, and Phi-3 from $\mathcal{M}$38 to $\mathcal{M}$39 (Jeong et al., 2 Feb 2026).

6. Limitations, failure modes, and open directions

The research literature is also unusually explicit about failure modes. A first recurring issue is weak-to-strong supervision. SRR-Judge improves weaker or comparable search agents but degrades DeepSeek-V3.1, with BrowseComp falling from $\mathcal{M}$40 to $\mathcal{M}$41, because a weaker judge can impose its own biases on a stronger agent (Zhang et al., 8 Feb 2026). More generally, step-level correction depends critically on verifier quality. In verifier-guided TTS, replacing Qwen2.5-Math-PRM-7B with math-shepherd-mistral-7b-prm sharply reduced Hybrid TTS performance on MATH500 (Chang et al., 21 Jul 2025). SSR likewise depends on accurate decomposition and confidence estimation; noisy Socratic steps can mislocalize the real error (Shi et al., 13 Nov 2025).

A second issue is over-refinement. Unconditional rewriting can damage already-correct steps or solutions. In verifier-guided TTS, unconditional Cover degraded Qwen2.5-7B on MATH500 from a small first-iteration gain to roughly $\mathcal{M}$42–$\mathcal{M}$43 by later iterations, whereas PRM_Cover rose steadily to $\mathcal{M}$44 (Chang et al., 21 Jul 2025). Self-Refine observed that some tasks are non-monotonic across iterations, which is why best-of-iteration selection matters (Madaan et al., 2023). ThinkTwice’s training dynamics show the same phenomenon in another form: refinement must eventually shift from rectification to preservation, or it damages correct solutions (Jiao et al., 2 Apr 2026).

A third issue is computational overhead. Step-level schemes multiply calls to judges, verifiers, or the base model itself. SSR requires decomposition, $\mathcal{M}$45 local re-solves, confidence estimation, and full-chain regeneration (Shi et al., 13 Nov 2025). SRR-Judge incurs an extra judge call at every search step and higher cost when $\mathcal{M}$46 candidate steps are generated (Zhang et al., 8 Feb 2026). S$\mathcal{M}$47c-Math’s data construction requires step-level sampling, pass@k verification, and reflection/improvement generation with a large model (Yan et al., 2024). SCMAPR introduces a multi-agent loop over routing, policy synthesis, rewriting, atomization, entailment checking, and revision (Yang et al., 7 Apr 2026). Internal Flow Signatures require replay for signature extraction, Jacobian-vector products for path integration, and suffix regeneration after rollback (Jeong et al., 2 Feb 2026).

A fourth issue is that step-level signals are still imperfect proxies for correctness. SRR-Judge’s best correlations are below $\mathcal{M}$48, limited by environmental stochasticity and hidden factors (Zhang et al., 8 Feb 2026). S$\mathcal{M}$49c-Math explicitly notes that it has no calibrated uncertainty or explicit verifier; correction is pattern-driven and can fail to trigger or over-trigger (Yan et al., 2024). Internal Flow Signatures perform better on QA and General hallucination settings than on Summarization and Dialogue, where errors are more diffuse and less tied to sharp depth-localized anomalies (Jeong et al., 2 Feb 2026).

Several research directions are already visible in the literature. SRR-Judge explicitly proposes folding the judge into the solver via an internal confidence score or a learned “step quality” head, and varying refinement depth according to score severity (Zhang et al., 8 Feb 2026). IPR points to more general step reward models and curricula that exploit reward magnitudes rather than only pairwise preferences (Xiong et al., 2024). Internal Flow Signatures suggest multi-site interventions instead of a single culprit block, especially for long-form tasks (Jeong et al., 2 Feb 2026). ThinkTwice notes that its framework natively supports arbitrary numbers of refinement turns, though only one is studied (Jiao et al., 2 Apr 2026). GSR suggests that learned refinement may generalize across model scales and domains once the conditioning structure is properly trained (Wang et al., 27 Aug 2025).

Taken together, these results suggest that conditional step-level self-refinement is evolving from prompt-level heuristic into a general control principle for reasoning systems: identify a locally verifiable intermediate unit, decide whether it is trustworthy, and apply the smallest intervention that changes the downstream trajectory only when necessary. The main unresolved questions concern verifier reliability, compute allocation, and how much of the correction policy can be internalized into a single model without sacrificing the localization fidelity that step-level methods currently obtain from explicit judges or verifiers.