Recursive-in-Recursive Self-Improvement (RiRSI)

• Recursive-in-Recursive Self-Improvement (RiRSI) is an advanced AI architecture where a system improves an object-level capability with a second, meta-level process that improves the evaluator or organizer of that improvement, incorporating recursive self-optimization and meta-evaluation techniques to dynamically enhance performance.
• An ordinary recursive self-improvement system updates models, datasets, or policies, while RiRSI adds a coupled loop that modifies the evaluation or optimization process, requiring an evolving state that includes data, tools, research infrastructure and mechanisms, leading to higher-order improvements.
• RiRSI has various implementations, including STOP and DataFoundry, and attempts meta-evaluators to improve search policies, data preparation, and agent harnesses, which increases efficiency in various coding, mathematical, and reasoning tasks, but it remains under a fixed body of evaluation signals and cannot guarantee anchorage.

Recursive-in-Recursive Self-Improvement (RiRSI) denotes an architecture in which a system improves an object-level capability while a second, meta-level process improves the machinery that performs, evaluates, or organizes that improvement. An ordinary recursive self-improvement loop may update a model, program, harness, dataset, memory, or policy according to a relatively fixed evaluator. RiRSI adds a coupled loop in which the evaluator, optimizer, search policy, data-generation process, research procedure, or improvement operator is itself modified. The term is not uniformly formalized across the literature; it is best treated as an overview of nested self-modification, evaluator co-evolution, meta-optimization, and iterative improvement of AI research processes.

1. Conceptual definition and scope

A first-order improvement loop can be represented as

$x_{t+1}=F(x_t,E),$

where $x_t$ is the current output, policy, model, program, harness, or artifact, and $E$ is a fixed evaluator. The system generates or modifies $x_t$, evaluates the result, and retains a candidate according to the evaluator.

RiRSI introduces additional evolving state:

$x_{t+1}=F(x_t,E_t,\mathcal{D}_t,\mathcal{T}_t),$

$E_{t+1}=G(E_t,x_t,\mathcal{D}_t,\mathcal{T}_t),$

where $\mathcal{D}_t$ represents data or curriculum generation and $\mathcal{T}_t$ represents tools, harnesses, training procedures, or research infrastructure. These equations are an interpretive abstraction of mechanisms discussed across the literature rather than a single canonical formalism (Chen et al., 8 Jul 2026).

A stronger higher-order system can also modify the update mechanisms themselves:

$$x_{t+1}=F_{\phi_t}(x_t,E_t), \qquad E_{t+1}=G_{\psi_t}(E_t,x_t),$$

$$\phi_{t+1}=H(\phi_t,x_t,E_t), \qquad \psi_{t+1}=K(\psi_t,x_t,E_t).$$

Here, $x_t$0 and $x_t$1 parameterize the mechanisms that improve the object and evaluator. A system becomes strongly recursive-in-recursive when the outer process changes the inner improvement process rather than merely repeating it.

The distinction is therefore between:

1. Object-level improvement: improving an answer, program, policy, model, dataset, memory, or harness.
2. Meta-level improvement: improving how object-level candidates are generated, evaluated, selected, scheduled, or trained.
3. Higher-order improvement: improving the meta-level process itself, including its optimizer, evaluator, objective, or research-direction mechanism.

The literature also distinguishes the source of the improvement signal. An anchored system uses an external reward, verifier, benchmark, scientific corpus, held-out task set, or human judgment. A goal-drifting system can alter the standard used to measure progress while retaining some external objective. A fully self-referential system derives its evaluation standard from internal processes. The latter is the most problematic because internal consistency no longer establishes correspondence with an external goal (Tang et al., 11 Sep 2026).

2. Historical and formal foundations

Early formalizations of recursive self-improvement model a finite program space rather than arbitrary self-modifying computation. A finite set of programs $x_t$2 is equipped with a score function $x_t$3, a current program $x_t$4, a candidate-generation mechanism, an acceptance rule, and a stopping criterion. The current program generates $x_t$5, and the update is

$x_t$6

Under a Markov restriction, each program defines a fixed distribution over candidate programs. The raw generation process is represented by a transition matrix $x_t$7, making it a homogeneous Markov chain; score-based rejection introduces self-loops in the accepted-program process. This formulation is a finite-state stochastic optimization process whose states are called programs, rather than a model of arbitrary source-code self-rewriting (Wang, 2018).

The score can be constructed as an expected hitting time to a unique target program $x_t$8:

$x_t$9

where $E$0 is the first time the process reaches $E$1. A constructive procedure assigns scores in nondecreasing order by repeatedly selecting the unassigned program with the smallest expected hitting time. The claimed complexity is

$E$2

where $E$3 and $E$4 is the total number of possible transitions. The construction is analogous in spirit to Dijkstra’s algorithm, except that the distance is an expected stochastic hitting time.

This formalism establishes computable, single-level iterative RSI under strong assumptions: finite program space, available transition distributions, a specified target, enumerable supports, computable hitting times, and an effectively available score. It does not include an improvement procedure that improves itself, a meta-optimizer, nested self-reference, or recursive modification of the score evaluator. Its recurrences describe Markov-chain expectations, not recursive calls among improvement procedures.

Generalized Agent Iteration (GAI) provides a broader abstraction. A system configuration is

$E$5

where $E$6 is the policy, $E$7 an action critic, $E$8 a modifier, $E$9 a modification critic, and $x_t$0 the evaluation base. The agent is the subset of components permitted to change. Ordinary generalized policy iteration has $x_t$1, while recursive self-improvement begins when $x_t$2, allowing the improvement mechanism to rewrite itself. If $x_t$3 is also modifiable, the system can exhibit goal drift or become fully self-referential. GAI formalizes the boundary between ordinary iterative policy improvement and RSI, but it does not provide an explicit hierarchy of modifiers $x_t$4 or a convergence theory for arbitrary nesting (Tang et al., 11 Sep 2026).

3. Implementations of nested improvement

Several systems instantiate different levels of RiRSI without altering the underlying foundation model.

Code-generation and scaffolding recursion

The Self-Taught Optimizer (STOP) begins with an improver $x_t$5 that uses a fixed LLM to generate and select improved downstream programs. The improver is then applied to its own source:

$x_t$6

where $x_t$7 evaluates an improver by its downstream performance. The LLM remains fixed; STOP recursively improves the scaffolding that prompts, samples, evaluates, and selects programs. It can discover beam search, genetic algorithms, simulated annealing, caching, adaptive sampling, and exploration–exploitation policies. Its improvements transferred from learning parity with noise to String Grid Distance, modified quadratic assignment, 3-SAT, MaxCut, and parity without noise. The method is recursive code-generation improvement, not weight-level RSI (Zelikman et al., 2023).

DataFoundry moves the mutable object upstream by evolving the data preparator rather than filtering its outputs after generation. A runtime specification $x_t$8 is compiled, used to generate a pilot set, assessed for instruction alignment, self-containment, information density, factual grounding, domain utility, and numerical correctness, and then revised through component-level adapters. The central loop is

$x_t$9

Its Skills-as-Modules architecture separates a compiler, contextualized-data assessor, and evolution-adapter designer. The preparator is improved before large-scale data production, and the resulting datasets are evaluated through downstream supervised fine-tuning. In the reported experiments, two evolution rounds generally improved results, while a third could destabilize the runtime (Yang et al., 30 Aug 2026).

Harness and agent-artifact evolution

Recursive harness systems modify the executable control layer surrounding a frozen model. Recursive Self-Evolving Agents (RSEA) maintain a three-layer natural-language state comprising an imperative strategy, reusable skills, and a procedural playbook. A frozen model produces trajectories using the current state; the same model rewrites the state; a disjoint held-out split determines whether the candidate is retained. The strict keep-better gate ensures that the deployed artifact does not underperform the current working state on validation, with fallback to vanilla ReAct if no candidate beats the empty state. RSEA improved ALFWorld performance from 64.6% for ReAct to 69.3% single-pass and 79.4% with retry, but did not improve the underlying model parameters or the selection process itself (Nguyen et al., 17 Jun 2026).

ModularRSI decomposes a coding-agent harness into Agent Loop, Tool Use, Observation Management, Context Management, and Task Completion Detection. It uses same-task successful–failed trajectory contrasts, aggregates diagnoses across tasks, maintains evolution history, evolves modules independently, and performs cross-module integration. Evolution tasks are benchmark-disjoint from downstream evaluation. The resulting harness improved TerminalBench 2.0 accuracy from 47.57% to 52.43% and SWE-Bench Verified accuracy from 73.40% to 76.45% in the reported setting. However, the diagnosis, validation, integration, and selection machinery remain externally fixed; the framework does not recursively improve its own evolution algorithm (Wu et al., 14 Sep 2026).

The Mendel Gödel Machine (MGM) extends archive-based scaffold evolution with comparative evidence. Clonal mutation uses one agent and one failed trajectory; reaction-norm mutation compares multiple trajectories of the same agent across tasks; cross-lineage hybridization compares different agents on a shared task. Agents, trajectories, outcomes, and lineages are stored in an archive. The new operators improve diagnostic candidate-set compression under a surrogate additive fitness model and produced higher coding performance than HGM on SWE-bench and Polyglot. MGM is repeated self-modification with evolutionary inheritance, not a formal system in which the self-modification algorithm itself is recursively improved (Liu et al., 7 Aug 2026).

Dream-RSI introduces a distinct meta-level: an object-level discovery agent improves programs, algorithms, mathematical constructions, or GPU kernels, while an exploration policy improves how search branches, batches, continues, and stops. Completed discovery trees become replay simulators. Candidate exploration policies are evaluated off-policy by revealing recorded branches and outcomes without invoking the coding agent or evaluator again. The selected policy is redeployed online, producing a new discovery tree and expanding the replay-world pool. The loop is

$x_{t+1}=F(x_t,E_t,\mathcal{D}_t,\mathcal{T}_t),$0

The system therefore improves the search process that generates object-level improvements, although the coding agent, evaluator, and policy-development model remain fixed (Zheng et al., 14 Sep 2026).

4. Parameter, memory, and training recursion

RiRSI does not require model-weight modification, but parameter-level systems provide a stronger form of coupling between object and meta-level learning.

PRefLexOR combines structured reasoning tokens, dynamically generated preference data, ORPO and EXO optimization, and inference-time reflection. Its inner recursion generates, critiques, and revises reasoning; its outer recursion uses revised outputs as preference-training material, thereby changing model parameters. The system remains anchored by a fixed scientific corpus, externally specified training procedure, and model-generated preference signals. Its implementation on approximately 3-billion-parameter models demonstrates recursive self-training and inference-time refinement, not unrestricted autonomous self-modification (Buehler, 2024).

RISE constructs a synthetic teacher from the model’s own reward-improving RLVR trajectory. An RLVR update produces $x_{t+1}=F(x_t,E_t,\mathcal{D}_t,\mathcal{T}_t),$1 from $x_{t+1}=F(x_t,E_t,\mathcal{D}_t,\mathcal{T}_t),$2; extrapolation creates a future policy; on-policy distillation moves the student toward that future policy:

$x_{t+1}=F(x_t,E_t,\mathcal{D}_t,\mathcal{T}_t),$3

The teacher is refreshed after every iteration. RISE therefore converts sparse outcome feedback into dense token-level supervision while repeatedly generating a new teacher from the latest improvement. It uses either weight-space or logit-space extrapolation, with the default extrapolation coefficient decaying from $x_{t+1}=F(x_t,E_t,\mathcal{D}_t,\mathcal{T}_t),$4 to $x_{t+1}=F(x_t,E_t,\mathcal{D}_t,\mathcal{T}_t),$5. The method improves mathematical reasoning, code generation, and multi-turn agents, but it does not modify the optimizer, evaluator, or improvement architecture (Li et al., 4 Sep 2026).

ScienceBuddy explicitly couples harness evolution and model reinforcement learning. The inner recursion modifies instructions, skills, and context-management procedures while holding model parameters fixed. The outer recursion trains the model using rubric-derived rewards while holding the selected harness and rubrics fixed. The coupled process is

$x_{t+1}=F(x_t,E_t,\mathcal{D}_t,\mathcal{T}_t),$6

In the reported biomedical setting, harness-only validation improved from 31.1% to 51.1% with model weights unchanged, model-only pass@4 increased from 48.3% to 67.8%, and the coupled study increased held-out single-attempt test accuracy from 42.2% to 73.3%. This is one of the clearest empirical examples of nested coupling between procedural and parameter-level improvement, although the auxiliary editor, rubrics, infrastructure, and outer optimization framework remain fixed (Xue et al., 15 Sep 2026).

RSIBench-Data isolates data-centric research. A researcher agent proposes training data $x_{t+1}=F(x_t,E_t,\mathcal{D}_t,\mathcal{T}_t),$7 and a bounded configuration $x_{t+1}=F(x_t,E_t,\mathcal{D}_t,\mathcal{T}_t),$8, trains a checkpoint $x_{t+1}=F(x_t,E_t,\mathcal{D}_t,\mathcal{T}_t),$9 from a fixed base model, receives controlled evaluation feedback, and revises its strategy. Later attempts improved over the first valid candidate in 14 of 24 settings, but 18 of 23 searches that continued after their best observed score ended below that peak. The benchmark therefore distinguishes discovery from reliable recursive improvement. It does not allow modification of the optimizer, architecture, evaluator, benchmark, serving stack, or researcher scaffold (Meng et al., 28 Jul 2026).

PAST-Bench studies persistent experience rather than weights. Agents operate across fresh-session episodes and are evaluated on whether they save, retrieve, apply, and update memory, skills, and procedures. Hermes+ adds researcher-designed interventions for planning-time consultation, typed memory bindings, procedural routing, retrieval-before-action, and synchronous overwrite. The benchmark establishes a memory-mediated first-level loop, but the interventions are not autonomously discovered by the agent (Xue et al., 4 Aug 2026).

5. Meta-optimization, evaluators, and governance

The central difficulty of RiRSI is that every loop makes a claim that its evaluation signal can substitute for human judgment. The strength of the loop is bounded by the reliability, independence, and external grounding of that signal (Chen et al., 8 Jul 2026).

Verification can be ordered from stronger to weaker forms:

1. Formal verifiers: proof checkers, type systems, and formal specifications.
2. Execution feedback: unit tests, compilers, runtime performance, and executable benchmarks.
3. Learned judges: reward models, process reward models, and LLM-as-a-judge.
4. Rubrics and structured criteria: interpretable but potentially evolvable and gameable.
5. Meta-evaluation: evaluating judges, reward models, rubrics, or evaluation procedures.
6. Intrinsic self-assessment: confidence, likelihood, self-consistency, and internal value signals.

Nested systems are vulnerable to self-confirming loops when the generator and evaluator share parameters or biases. They can also exhibit reward hacking, specification gaming, model collapse, diversity collapse, evaluator drift, loss of grounding, and methodological amnesia. Improving an evaluator can be more dangerous than improving a policy because the system may increase measured performance by changing what counts as success.

SAHOO addresses ordinary iterative improvement through a Goal Drift Index (GDI), constraint-preservation checks, and regression-risk quantification. GDI combines semantic, lexical, structural, and distributional drift relative to an immutable baseline. Constraint Preservation Score measures satisfaction of explicit predicates, while regression risk estimates whether later cycles will fall below the historical maximum. Its experiments on code, truthfulness, and mathematics reported gains of 18.3% in code and 16.8% in mathematical reasoning, with lower truthfulness gains and nonzero violations. SAHOO is a monitoring and control layer, not a recursively self-improving monitor: it does not modify its own evaluator, constraints, stopping rules, or safeguards (Sahoo et al., 6 Mar 2026).

RSEA similarly emphasizes that candidate generation is easier than safe retention. Its held-out gate prevents many observed regressions but does not establish universal test-set safety. DataFoundry uses stable component interfaces, pilot validation, smoke tests, adapter isolation, and bounded rounds. ModularRSI uses AST validation, protocol compliance, diff review, execution checks, historical evolution records, and benchmark-disjoint evaluation. These systems share a common pattern:

$E_{t+1}=G(E_t,x_t,\mathcal{D}_t,\mathcal{T}_t),$0

Strong RiRSI systems therefore require immutable or externally enforced evaluators, disjoint validation data, rollback, versioned artifacts, provenance, regression testing, bounded modification scopes, and explicit separation between model proposals and authority-bearing adjudication.

MetaRSI formalizes this architecture most directly. The deployed system is

$E_{t+1}=G(E_t,x_t,\mathcal{D}_t,\mathcal{T}_t),$1

with data, model, and harness states. Data-RSI, Harness-RSI, and Model-RSI modify separate write surfaces. A Level-2 scheduler chooses operator order and revises operator proposal policies; a Level-3 meta-agent modifies the scheduler. Protected code retains authority over schema validation, execution isolation, evaluation, selection, release, and lineage recording. The hierarchy is therefore:

$E_{t+1}=G(E_t,x_t,\mathcal{D}_t,\mathcal{T}_t),$2

The reported MetaRSI system obtained an average gain of +10.9 over a frozen system, compared with +7.3 for fixed $E_{t+1}=G(E_t,x_t,\mathcal{D}_t,\mathcal{T}_t),$3 composition and +4.5 for random admissible composition. Across five terms, the meta-updated branch reached +26.2 compared with +20.7 without meta-update. These results demonstrate improvement of the organization of self-improvement under protected evaluation, not unrestricted modification of the objective or evaluator (Tan et al., 6 Sep 2026).

6. Dynamics, empirical limits, and open problems

A system can improve rapidly without being recursively self-amplifying. The distinction is captured by models of AI-enabled research and development. Let $E_{t+1}=G(E_t,x_t,\mathcal{D}_t,\mathcal{T}_t),$4 denote a resource-adjusted capability coordinate, $E_{t+1}=G(E_t,x_t,\mathcal{D}_t,\mathcal{T}_t),$5 the potential recursive gain, $E_{t+1}=G(E_t,x_t,\mathcal{D}_t,\mathcal{T}_t),$6 operational closure, and $E_{t+1}=G(E_t,x_t,\mathcal{D}_t,\mathcal{T}_t),$7 frontier hardness. Realized recursive gain is

$E_{t+1}=G(E_t,x_t,\mathcal{D}_t,\mathcal{T}_t),$8

The recursive reproduction number is

$E_{t+1}=G(E_t,x_t,\mathcal{D}_t,\mathcal{T}_t),$9

When $\mathcal{D}_t$0, local capability perturbations amplify across development cycles; when $\mathcal{D}_t$1, they decay. This is a local stability criterion, not a theorem of indefinite intelligence growth. Baseline research productivity can increase the speed of progress without changing criticality, while long development-cycle delays can make a supercritical regime appear slow. Frontier hardening can eventually return a system to a subcritical regime (Burtsev, 31 Aug 2026).

Economic models similarly represent recursive feedback through directed production graphs. A feedback path’s strength is the product of its edge elasticities. For algorithmic efficiency $\mathcal{D}_t$2, self-sustaining acceleration requires

$\mathcal{D}_t$3

A core AI-capability loop has strength

$\mathcal{D}_t$4

while broader economic systems add feedback through output, compute, data, and training resources. These models distinguish narrow capability improvement from broad economically valuable capability and do not explicitly represent a separate state for the system’s ability to improve the process that improves AI (Cunningham et al., 14 Sep 2026).

The principal empirical limitations across current systems are:

• Evaluator dependence: self-generated or model-based judgments can reinforce errors.
• Distribution shift: improvements selected on development tasks may fail on held-out or newly generated tasks.
• Non-monotonicity: later self-modifications frequently undo earlier gains.
• Search variance: a strong candidate may arise from stochastic training or a lucky data mixture rather than a learned general principle.
• Capability preservation: target-task gains may coexist with regressions elsewhere.
• Cost escalation: nested loops multiply inference, training, evaluation, and environment-interaction costs.
• Replay limitations: historical simulators cannot evaluate unsupported novel branches.
• Memory pollution: persistent artifacts can preserve stale, incorrect, or overgeneralized rules.
• Evaluator co-adaptation: a system may learn to satisfy a judge rather than the intended objective.
• Lack of arbitrary-depth evidence: most systems demonstrate only one or two nested levels and finite iteration horizons.
• No proof of open-ended growth: observed gains do not establish indefinite acceleration, general intelligence, or an intelligence explosion.

A genuine higher-order system would need persistent and transferable meta-policies, independently grounded evaluation at each level, explicit state for object-level and meta-level improvement, nested resource-bounded semantics, mechanisms for causal attribution, held-out evaluation of the improvement process, capability-preservation constraints, and protections against modifying the evaluator or release gate. It would also need to demonstrate that improvements in the meta-level process increase the rate, reliability, breadth, or quality of object-level improvement across new tasks and environments.

The current literature supports a layered conclusion. Finite-state stochastic RSI provides a formal baseline but not self-improving optimizers. STOP, RSEA, MGM, ModularRSI, DataFoundry, Dream-RSI, and RSIAgent demonstrate artifact-, scaffold-, memory-, data-, or exploration-policy improvement while keeping important system components fixed. PRefLexOR and RISE show recursive training or inference-time refinement, with externally anchored preference or reward signals. ScienceBuddy demonstrates coupled harness and model improvement. MetaRSI most explicitly implements a meta-level scheduler that improves the organization of several RSI operators. GAI provides a general vocabulary for determining which components are modifiable and whether evaluation remains externally grounded.

None of these results establishes unrestricted, arbitrary-depth recursive-in-recursive self-improvement. The strongest demonstrated interpretation is bounded nested optimization: an inner process improves a target system, while an outer process improves the data, scaffold, evaluator interface, search policy, schedule, or research procedure that governs the inner process. The decisive unresolved problem is not merely whether systems can generate candidate improvements, but whether they can improve and independently validate the machinery that determines what counts as improvement without losing grounding, diversity, capability preservation, or control.