Test-time Recursive Thinking: Self-Improvement without External Feedback

Abstract: Modern LLMs have shown rapid improvements in reasoning capabilities, driven largely by reinforcement learning (RL) with verifiable rewards. Here, we ask whether these LLMs can self-improve without the need for additional training. We identify two core challenges for such systems: (i) efficiently generating diverse, high-quality candidate solutions, and (ii) reliably selecting correct answers in the absence of ground-truth supervision. To address these challenges, we propose Test-time Recursive Thinking (TRT), an iterative self-improvement framework that conditions generation on rollout-specific strategies, accumulated knowledge, and self-generated verification signals. Using TRT, open-source models reach 100% accuracy on AIME-25/24, and on LiveCodeBench's most difficult problems, closed-source models improve by 10.4-14.8 percentage points without external feedback.

• The paper introduces TRT, a meta-inference framework that allows LLMs to iteratively refine answers using strategy-conditioned rollouts and reflective error analysis.
• It demonstrates significant improvements, achieving 100% accuracy on AIME benchmarks and increasing code generation accuracy by up to 14.8 percentage points.
• The approach scales across LLM sizes and domains by shifting focus from parallel sampling to recursive self-improvement with a compact negative knowledge base.

Test-time Recursive Thinking: Self-Improvement without External Feedback

Overview and Motivation

The paper "Test-time Recursive Thinking: Self-Improvement without External Feedback" (2602.03094) introduces Test-time Recursive Thinking (TRT), a meta-inference framework enabling LLMs to self-improve their answers and reasoning quality during inference without access to external feedback or ground-truth rewards. TRT directly targets the challenge of improving solution quality on complex reasoning and coding benchmarks by iteratively running a three-stage loop: (1) knowledge-conditioned diverse solution generation, (2) self-ranking and self-verification without supervision, and (3) reflective error analysis to update a compact knowledge list which constrains search and informs future strategies.

This approach differs fundamentally from classical ensemble methods, tree-of-thought search, and parallel aggregation in that it explicitly accumulates reusable negative constraints ("don'ts") and leverages strategy diversification, enabling non-trivial trajectory improvements beyond what is reachable by parallel sampling alone. The practical effect is demonstrated on mathematics (AIME) and competitive programming (LiveCodeBench), as well as across open- and closed-weight LLMs.

Methodology: The TRT Algorithm

TRT applies a recursive self-improvement loop at test time, structured as follows:

1. Strategy-Conditioned Rollout Generation: For each problem, $K$ rollouts are generated in round $t$, each receiving a strategy prompt that is adaptively constructed based on the current knowledge list. Strategies are dynamically constructed (e.g., algorithmic paradigm, proof type); this drives exploration of the solution space orthogonally to naive diversity.
2. Self-Selection and Verification: Among the rollouts, a self-judgment phase selects a provisional best candidate. In code, this leverages self-generated test cases and execution feedback rather than pre-supplied answers. In math, mutual exclusivity in integer responses is exploited.
3. Error Reflection and Knowledge Accumulation: All non-best solutions are pairwise-compared to the selected one; from these comparisons, the model extracts and phrases negative constraints capturing failure modes, edge cases, or undesirable patterns. These insights are appended to the knowledge list, which in turn guides subsequent strategy synthesis.

   Figure 1: The TRT pipeline cycles through knowledge-conditioned generation, self-ranking, and reflective error analysis to build an evolving, reusable knowledge base.

A critical aspect is that knowledge is stored in highly abstract and compact form, typically negative constraints or distilled corrective advice, preserving context efficiency. The knowledge list is pruned as needed but rarely grows to more than a few percent of model context for even long runs.

Empirical Results: Reasoning and Code Generation

Mathematical Reasoning: AIME

TRT achieves 100% accuracy on AIME-25 and AIME-24 with both gpt-oss-120b and Qwen3-235B. This is accomplished through 64 rounds of knowledge accumulation, with performance improving monotonically and majority voting over previous answers accelerating convergence. Notably, this surpasses the best majority-vote and parallel sampling baselines at equivalent or greater compute.

Figure 2: TRT yields monotonic accuracy improvement on AIME-25, outperforming parallel aggregation at matched sample count.

Code Generation: LiveCodeBench

Closed-source models (o4-mini, o3) evaluated on the most challenging LiveCodeBench v6 problems demonstrate substantial monotonic gains with TRT. o4-mini's accuracy for selected solutions improves from 63.5% to 73.9% (+10.4 pp) over 8 TRT rounds, while o3 jumps from 57.1% to 71.9% (+14.8 pp). Both results exceed RSA (recursive self-aggregation), highlighting the benefit of cross-round knowledge carryover versus pure population-based aggregation.

Figure 3: TRT outperforms RSA in iterative code generation accuracy improvements, even without access to external feedback.

TRT's exploration is shown to be semantically richer: at fixed sample count, pass@k is elevated by 2-7 percentage points over temperature-1 random sampling, indicating that knowledge- and strategy-informed generation explores qualitatively distinct solutions.

Figure 4: Strategic exploration with TRT increases pass@k versus independent sampling, confirming value in knowledge-guided search.

Ablations and Analysis

Exploration Versus Breadth

TRT's gains are shown to be primarily a function of depth (number of recursive rounds) rather than parallel width (number of rollouts per round); ablations confirm negligible improvement when increasing width beyond two rollouts per round, once sufficient depth is achieved.

Knowledge List Efficiency

Despite iterative accumulation, the knowledge list remains extremely compact, typically comprising less than 1.5% of model context for 64 rounds in math and under 0.35% for code problems after 8 rounds. This enables TRT to scale to long-horizon tasks without context exhaustion.

Figure 5: The knowledge list length remains compact, avoiding context bloat.

Knowledge Representation and Efficacy

A critical design validation is that storing negative ("don't") knowledge yields higher final accuracy than merely accumulating successful steps ("dos"). Reflective analysis shows that the learned knowledge clusters for code are dominated by performance, edge cases, and indexing bugs, a clear indication that TRT captures actionable generalizations over the course of iterative self-improvement.

Figure 6: Negative knowledge (don'ts) consistently improves performance on math tasks compared to positive-only strategies.

Figure 7: In code, TRT-learned knowledge emphasizes performance, edge cases, and indexing, aligning with dominant bug classes in competitive programming.

Sequential Edit Superiority

Experimental evidence demonstrates that sequentially editing prior solutions informed by knowledge ("Edit") consistently outperforms full regeneration from scratch ("Regenerate") for each round. This underscores the importance of incremental correction, a key distinction from methods relying exclusively on parallel fresh generations.

Figure 8: Sequential knowledge-driven edits outperform naive regeneration, indicating the benefit of incremental improvement.

Implications and Future Directions

TRT demonstrates that LLMs can, via context construction and in-context knowledge accumulation, recursively improve solution quality on complex reasoning tasks at test time without any external supervision or learned reward model. The framework is extensible across LLM scales—even 4B parameter models show strong iterative improvement (Figure 9)—and diverse domains, as evidenced by its generalization to both AIME 2024 and LiveCodeBench.

Figure 9: TRT's self-improvement benefit generalizes across model scale, including smaller LLMs.

Figure 10: TRT generalizes to AIME-24, showing matching improvement patterns as on AIME-25.

In practice, TRT mitigates the stagnation and redundancy observed in parallel ensemble methods, suggesting a paradigm shift in how LLM computational budget should be allocated at inference time—not increased width, but increased recursive depth and knowledge abstraction. TRT's explicit reflective mechanism also creates new opportunities for AI-assisted knowledge discovery, beyond merely increasing pass rates.

Further, as LLM architectures with broader context windows and higher compositionality emerge, TRT provides a blueprint for embedding continual adaptation and self-critique without offline retraining. Future developments could include multi-problem knowledge transfer, more systematic knowledge condensation, or integration with RL for meta-learning recursive thinking protocols. There are also direct implications for scientific discovery workflows, where accumulating negative results (failures) is critical for efficient hypothesis pruning.

Conclusion

"Test-time Recursive Thinking: Self-Improvement without External Feedback" establishes TRT as an effective strategy for test-time, in-context self-improvement in LLMs, markedly enhancing performance on both reasoning and code synthesis tasks without the need for labels or external critique. By leveraging novel mechanisms for knowledge accumulation, strategy diversification, and reflection-driven error abstraction, TRT enables sample-efficient, monotonic solution quality improvements that surpass parallel aggregation and single-pass self-refinement. These results have immediate implications for LLM deployment, inference-time optimization, and offer a robust foundational paradigm for developing more autonomous, self-correcting AI systems.