Create and explore imaginative 3D worlds with our new product Dreamfield (opens in a new tab)

Causal factors behind performance gains in RL-based reasoning models

Determine whether the performance enhancements observed in reinforcement learning–trained Large Reasoning Models (LRMs) are primarily caused by (i) increased exposure to established mathematical benchmark data during training, (ii) the greater inference-time compute allocated to thinking tokens, or (iii) genuine reasoning capabilities developed through reinforcement learning, by isolating and quantifying the contribution of each factor under controlled experimental conditions.

Background

The paper compares thinking models (e.g., Claude-3.7-Sonnet-Thinking, DeepSeek-R1) with their non-thinking counterparts under matched inference compute and observes comparable pass@k on MATH500, but widening gaps on AIME24 and AIME25. These observations raise interpretive challenges about whether differences stem from task complexity, data contamination, or inference compute allocation.

To address confounds present in established math benchmarks, the authors employ controllable puzzle environments that allow manipulation of complexity and inspection of intermediate reasoning traces. Nevertheless, identifying the causal drivers of observed performance improvements in LRMs remains an explicit unresolved question.

References

Currently, it is not clear whether the performance enhancements observed in recent RL-based reasoning (thinking) models are attributable to increased exposure to established mathematical benchmark data, to the significantly greater inference compute allocated to thinking tokens, or to reasoning capabilities developed by RL-based training?

— The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity  (2506.06941 - Shojaee et al., 7 Jun 2025) in Section 3 (Math and Puzzle Environments), opening paragraph

Whether the additional token use contributes causally to the success increase cannot be determined from these aggregate measurements.

— DeltaML-Bench: Evaluating Machine Learning Agents on Real-World Research Repositories  (2608.19653 - Moukpe et al., 20 Aug 2026) in Section Efficiency and Information Processing Cost

Three explanations compete: clipping that amplifies the model's prior for code-style reasoning; contamination, since from 60\,\% of each problem Qwen2.5-Math-7B reconstructs 54.60\,\% of MATH-500 against 3.80\,\% for Llama3.1-8B and only correct rewards give stable gains on a clean benchmark; and model strength, since the clipping bias is too small to carry a signal and gains appear on non-Qwen models too. The studies differ in prompts, clipping, and models, so the question is open, but they agree that a strong prior can override the $J$ analysis and that benchmarks older than the model can make any reward look useful.

— Checking the Checkers: A Synthesis of Verifier Noise and Robustness in RLVR  (2610.17824 - Gupta et al., 13 Oct 2026) in Section 4.5, “The Spurious-Reward Debate”; Section 7.2, “Resolved and Open Disagreements”