Recursive Criticism and Improvement

• Recursive Criticism and Improvement is a framework where AI systems iteratively critique and refine their outputs using introspection, self-evaluation, and self-modification.
• The methodology leverages formal models such as fixed-point theorems and bounded recursive loops to drive error correction and compounding performance gains.
• Practical applications include improved accuracy in reasoning, coding, and decision-making, with built-in safeguards to prevent runaway self-modification and ensure stability.

Recursive criticism and improvement denotes a family of mechanisms in which an AI system generates an answer, reasoning trace, policy, skill, or code artifact, critiques that artifact, and then uses the critique to produce a revised version, potentially repeating the cycle multiple times. In contemporary work, the term covers single-output refinement loops, learned critic-actor training schemes, multi-agent error-correction pipelines, bounded reflective architectures, and stronger notions of recursive self-improvement grounded in self-reference and fixed-point theorems. Across these variants, the central technical questions are how critique is represented, how improvement is scored, when recursion yields compounding gains rather than saturation or degradation, and what architectural and safety conditions are required for the loop to remain stable (Zhang et al., 5 Jul 2026).

1. Definitions, state spaces, and formal models

A strong formulation of recursive criticism and improvement is given in terms of introspection: a computational system’s capacity to build, execute, and evaluate an internal model of its own processing. In that formulation, an introspective AI must satisfy four functional properties: self-modeling, self-simulation, self-evaluation, and self-modification. The same framework treats these properties as the precondition for sustainable recursive self-improvement, because without them self-modification is blind to downstream effects; only beyond the “introspection threshold” does an iteration $S_n \to S_{n+1}$ yield genuine, compounding improvement rather than a plateau at a suboptimal fixed point (Zhang et al., 5 Jul 2026).

A restricted but precise formalization appears in work on recursive self-improving systems over a finite program set $P$. There, $S: P \to \mathbb{R}\cup\{\infty\}$ is a total score function in which lower values are better, and each program $p \in P$ induces a fixed distribution $w_p$ over candidate successors. The update rule is memoryless: sample $p' \sim w_{p_t}$ and replace $p_t$ with $p'$ only if $S(p') < S(p_t)$. Under the paper’s restricted assumptions, $S(p)$ is constructed as the expected number of self-improvement steps needed to reach a unique target program $P$0, and the resulting oracle is finite and computable in $P$1 time. In simulated random-graph settings with $P$2, the expected number of steps grew roughly linearly in $P$3, with a fitted line $P$4 and $P$5 over $P$6 (Wang, 2018).

A different formalization treats recursive improvement as a finite-horizon decision process over text trajectories. RISE converts a single-turn prompt into a deterministic MDP $P$7 in which the state concatenates the original prompt, prior model responses, and fixed introspection instructions; actions are textual responses; transitions append a new response plus the next introspection prompt; and reward is the sparse binary signal $P$8. This formulation shifts recursive criticism from an informal prompting heuristic to a multi-turn policy-optimization problem with on-policy rollouts and reward-weighted regression (Qu et al., 2024).

2. Fixed points, self-reference, and the introspection threshold

The strongest theoretical account of recursive criticism and improvement in current LLM literature is built on Kleene’s Second Recursion Theorem. In one formulation, for any total computable function $P$9, there exists an index $S: P \to \mathbb{R}\cup\{\infty\}$0 such that $S: P \to \mathbb{R}\cup\{\infty\}$1. By taking

$S: P \to \mathbb{R}\cup\{\infty\}$2

the theorem guarantees an index $S: P \to \mathbb{R}\cup\{\infty\}$3 for which

$S: P \to \mathbb{R}\cup\{\infty\}$4

so execution of $S: P \to \mathbb{R}\cup\{\infty\}$5 is equivalent to simulating itself for $S: P \to \mathbb{R}\cup\{\infty\}$6 steps. Extending this construction with an evaluator $S: P \to \mathbb{R}\cup\{\infty\}$7 and a modifier $S: P \to \mathbb{R}\cup\{\infty\}$8 yields

$S: P \to \mathbb{R}\cup\{\infty\}$9

and again by the theorem there exists a fixed point $p \in P$0 such that

$p \in P$1

In that sense, the paper formalizes a self-simulate $p \in P$2 self-evaluate $p \in P$3 self-rewrite loop as a computable fixed point of the operator $p \in P$4 (Zhang et al., 5 Jul 2026).

The same treatment states three necessary and sufficient conditions for this fixed-point construction: total computability of $p \in P$5, $p \in P$6, and $p \in P$7; existence of a universal simulator; and unbounded recursion, so that the architecture can support an infinite improvement trajectory $p \in P$8. On this account, standard feedforward Transformers are a poor substrate for strong recursive self-improvement because they are described as being in uniform $p \in P$9 and therefore unable to implement general fixed-point iteration or unbounded recursion (Zhang et al., 5 Jul 2026).

This argument is used to separate true introspection from the weaker empirical phenomenon of quasi-introspection. Current LLMs exhibit fragments of the four functional properties—statistical confidence calibration, learned “unknown” detectors, rough prediction of their own token probabilities, metacognitive projections, and shallow prompt or code rewriting—but they lack complete self-access, lack a discrete mirrored representation of their own weights, and operate in a feedforward regime without architectural recurrence. The same paper also identifies continuous parameters as an obstacle to exact symbolic self-reference of the Quine-style sort (Zhang et al., 5 Jul 2026).

Architectural proposals intended to cross this threshold include an externalized self-model or “self-card,” recurrent or iterative designs such as Universal Transformers with adaptive recurrence and Perceiver-style bottlenecks with iterative attention, memory-augmented agents that preserve self-state histories across episodes, and an approximate reflexive structure centered on a “synergistic core” subnetwork. These proposals are presented not as proofs of sufficient implementation, but as ways to recover a discrete self-description domain, unbounded self-simulation, and causal grounding for self-reports (Zhang et al., 5 Jul 2026).

3. Critique as an optimization signal

A major line of work operationalizes recursive criticism by training critics whose value is measured by downstream refinement rather than by the plausibility of the critique text itself. RCO introduces Critique Utility,

$w_p$0

estimated from judged refinements of an initial answer $w_p$1. The critic is then trained with an offline regularized RL objective,

$w_p$2

and recursive use is explicit: each refined output becomes the next stage’s initial response. Across five tasks, RCO-trained critics increased average CU by 8–12 points out of 100 and Response Quality Score by approximately 0.3–0.5 on a 1–10 scale; on six QA benchmarks, two math benchmarks, and two code benchmarks, refinements improved exact accuracy by 5–20% absolute over both the base actor and a direct-preference-trained critic. When chaining $w_p$3 rounds, RCO steadily increased Response Quality Score by approximately $w_p$4 each round, whereas self-refinement or a DPO-trained critic plateaued or degraded after round 2 (Yu et al., 27 Jun 2025).

RefCritic pushes the same idea into long chain-of-thought critics. Its critic model emits a long analysis $w_p$5, a binary correctness judgment, and free-text refinement feedback, and is optimized with dual rule-based rewards: instance-level correctness of the judgment and the refinement accuracy of the policy model after consuming the feedback. This directly targets the earliest failure mode of supervised critics identified in the paper: high critique accuracy with shallow reasoning and vague suggestions. On AIME25, Pass@1 improves from 14.4% to 21.2% for Qwen2.5-14B and from 49.2% to 56.3% for R1-Qwen-14B, while output length grows from approximately 500 to 3500 tokens for Qwen-14B and from approximately 3000 to 8000 tokens for R1-14B during RL training. Despite being trained only with solution-level supervision, the critic also improves ProcessBench F1 to 68.0 and 77.1 for the two backbones (Tang et al., 20 Jul 2025).

AvR treats refinement as a sequence of reward-bearing actions rather than a single terminal answer. It first learns answer $w_p$6 criticism $w_p$7 improvement transitions, then uses the trained model in a greedy recursive loop to synthesize long refinement chains that can themselves be used for SFT and length-controlled DPO. On AlpacaEval 2.0, the Stage 2 system reports 51.0% win rate and 51.4% length-controlled win rate, and the paper states that only 3k long CoTs already yield a +20% absolute win-rate gain (Zhang et al., 6 Jun 2025).

Deep Improvement Supervision provides a non-LLM contrast case. It interprets latent recursion in Tiny Recursive Models as classifier-free guidance and implicit policy improvement, assigns an explicit target to each loop, removes learned halting, and reduces the total number of forward passes from approximately 336 to 18, a ratio of about $w_p$8. With 0.8M parameters, DIS reports 24.0% accuracy on ARC-1 while maintaining quality comparable to standard TRMs (Asadulaev et al., 21 Nov 2025).

4. Runtime recursion, multi-agent correction, and self-modifying agents

Recursive criticism can also be implemented as a runtime capability rather than only as a training objective. RISE fine-tunes LLMs on on-policy multi-turn introspective trajectories, using either stronger-model distillation or self-distillation, so that the model learns to alter its response after prior failed attempts. On GSM8K without oracle stopping and evaluated at turn 5, LLaMa2-7B improves from 32.9% at turn 1 to 50.7% after two RISE iterations, Mistral-7B improves from 33.7% to 59.2% in one iteration of self-distillation, and GPT-3.5 obtains only a +4.6 point gain over five turns with prompting alone (Qu et al., 2024).

ICRL makes the solver and critic co-evolve from a shared backbone. The critic is rewarded by how much the solver improves after using its critique, while a distribution-calibration reweighting ratio

$w_p$9

down-weights improvements that depend too heavily on critique-conditioned behavior, and role-wise group advantage estimation stabilizes joint training. With Qwen3-4B and Qwen3-8B backbones, ICRL reports average gains of 6.4 points over GRPO on agentic tasks and 7.0 points on mathematical reasoning, and its learned 8B critic is described as comparable to 32B critics while using substantially fewer tokens (Lin et al., 13 May 2026).

Test-time Recursive Thinking shows that recursion need not rely on retraining or external labels. TRT alternates rollout generation, self-verification, contrastive reflection, and accumulation of a compact knowledge list $p' \sim w_{p_t}$0 of distilled failure modes. On AIME-25 and AIME-24, open-source models reach 100% accuracy with 64 rounds; on LiveCodeBench v6 Hard, closed-source models improve from 63.5% to 73.9% and from 57.1% to 71.9% after 8 rounds, and the knowledge list remains below 1.5% of a 128K context after 64 rounds (Zhuang et al., 3 Feb 2026).

In table reasoning, Table-Critic instantiates recursion as a collaborative multi-agent pipeline: Judge, Critic, Refiner, and Curator. The Judge identifies the first error and routes through a self-evolving template tree; the Critic localizes the faulty step and generates a focused critique; the Refiner regenerates the remainder of the chain from the correct prefix; and the Curator updates critique templates after successful correction. On WikiTQ, the reported Error Correction Rate is $p' \sim w_{p_t}$1, the Solution Degradation Rate is $p' \sim w_{p_t}$2, and the Net Gain is $p' \sim w_{p_t}$3; accuracy rises sharply in the first 2–3 iterations before plateauing, motivating $p' \sim w_{p_t}$4 as the convergence budget (Yu et al., 17 Feb 2025).

More radical systems apply recursive criticism to the agent’s own improvement machinery. Gödel Agent packages the current task policy $p' \sim w_{p_t}$5 and self-modification engine $p' \sim w_{p_t}$6 into a single state, exposes both through SELF_INSPECT, proposes code patches by LLM prompting, applies them by monkey-patching, and empirically verifies whether utility improved before committing the rewrite. MetaSkill-Evolve extends the same principle to branch-local skill files and branch-local meta-skills $p' \sim w_{p_t}$7, with a fast loop that evolves the task skill and a slow loop that rewrites the very Analyzer, Retriever, Allocator, Proposer, and Evolver prompts. On held-out test sets, MetaSkill-Evolve improves over the raw backbone by +23.54 points on OfficeQA, +16.09 on SealQA, and +1.92 on ALFWorld (Yin et al., 2024, Wang et al., 6 Jul 2026).

5. Oversight, boundedness, and control

Recursive criticism and improvement has an immediate alignment dimension because the same machinery that corrects errors can also rewrite safety constraints. One account states the main risks as runaway self-modification, utility-function security, and the leakproofing problem, then proposes hard-coded safety invariants in the initial self-model, guarded introspection frameworks, tiling agents, bounded RSI designs, and formal verification of each rewrite in the style of the Gödel Machine’s proof search (Zhang et al., 5 Jul 2026).

SAHOO systematizes this concern into measurable safeguards. It introduces a learned Goal Drift Index combining semantic, lexical, structural, and distributional signals; a Constraint Preservation Score over safety-critical predicates; and a regression-risk estimator that halts recursion when future deterioration becomes too likely. Across 189 tasks in code generation, mathematical reasoning, and truthfulness, SAHOO reports 18.3 percent improvement in code tasks, 16.8 percent in reasoning, and 3.8 percent in truthfulness; constraint preservation is 1.00 in code and reasoning and 0.9874 in truthfulness, 91.5% of tasks converge by cycle approximately 8.8, and regression events are rare at less than 1% of cycles excluding one outlier (Sahoo et al., 6 Mar 2026).

Recursive self-critiquing has also been proposed as a scalable oversight mechanism for systems that exceed direct human evaluability. The underlying hypotheses are that critique of critique can be easier than critique itself and that this relationship recurses. In human-human experiments, Gaokao Math accuracy rises from 66.3% at the response level to 82.5% at $p' \sim w_{p_t}$8, 90.6% at $p' \sim w_{p_t}$9, and 93.8% at $p_t$0 on a subset; in human-AI settings, humans reviewing Qwen outputs also improve over direct answer reading. The same paper, however, reports that AI-AI recursion produces only small relative improvements and that gains diminish beyond 3–4 levels while compute cost grows as $p_t$1 (Wen et al., 7 Feb 2025).

Boundedness appears both at the architectural level and at the systems-theoretic level. AERA enforces thresholds on term likelihood and model reliability, phases out or deletes low-reliability models, and maintains a global CPU/time budget

$p_t$2

At a larger scale, a continuous-time recursive-improvement model defines capability dynamics $p_t$3 and identifies the total feedback elasticity

$p_t$4

In that framework, superlinear runaway occurs iff $p_t$5, while nonsingularity can be certified when $p_t$6 and $p_t$7; proposed controls include power caps, throughput throttling, control-barrier-function constraints, and evaluation gates triggered when the rolling slope estimate crosses a safety margin (Nivel et al., 2013, Jafari et al., 8 Nov 2025).

6. Misconceptions, limits, and open problems

A persistent misconception is that any iterative self-refinement loop constitutes recursive self-improvement in the strong sense. The introspection-threshold view rejects that equation: current LLMs are described as quasi-introspective rather than truly introspective because they lack complete self-access, lack exact symbolic self-reference, and remain constrained by feedforward computation (Zhang et al., 5 Jul 2026).

A second misconception is that a good critic is merely one with high critique accuracy or human-preferred feedback. Several frameworks explicitly argue otherwise. RCO ties reward to actual actor improvement through Critique Utility, RefCritic adds refinement reward to judgment reward, and ICRL rewards the critic by the solver’s subsequent performance gain while explicitly addressing critique-conditioned distribution shift. This suggests that recursive criticism is fundamentally an intervention problem, not only an evaluation problem (Yu et al., 27 Jun 2025, Lin et al., 13 May 2026).

The literature also converges on recurrent practical limits. Table-Critic reports that iteration-wise accuracy rises sharply in the first 2–3 iterations before plateauing; recursive self-critiquing reports diminishing returns beyond 3–4 meta-levels; RefCritic identifies substantial compute cost in the dual-reward RL loop; MetaSkill-Evolve notes that its benchmarks are curated and relatively clean and that pipeline roles and wiring remain hand-designed; the finite-state RSI model relies on a precomputed oracle over all programs and a single absorbing objective; and the continuous-time singularity framework depends on the chosen capability metric and on regularity diagnostics (Yu et al., 17 Feb 2025, Wen et al., 7 Feb 2025, Tang et al., 20 Jul 2025, Wang et al., 6 Jul 2026, Wang, 2018, Jafari et al., 8 Nov 2025).

Open problems therefore remain at several levels. Strong recursive self-improvement still lacks an implementation that simultaneously satisfies self-modeling, self-simulation, self-evaluation, and self-modification in a substrate capable of unbounded recursion. Output-level recursive refinement is already empirically useful, but the transition from externally scaffolded loops to internalized, self-stabilizing introspection remains incomplete. The most plausible implication is that future progress will depend less on adding more critique turns per se than on improving self-access, verification, boundedness, and the fidelity with which critique-induced gains are converted into durable capability.