Asking the Right Questions: Improving Reasoning with Generated Stepping Stones

Abstract: Recent years have witnessed tremendous progress in enabling LLMs to solve complex reasoning tasks such as math and coding. As we start to apply LLMs to harder tasks that they may not be able to solve in one shot, it is worth paying attention to their ability to construct intermediate stepping stones that prepare them to better solve the tasks. Examples of stepping stones include simplifications, alternative framings, or subproblems. We study properties and benefits of stepping stones in the context of modern reasoning LLMs via ARQ (\textbf{A}king the \textbf{R}ight \textbf{Q}uestions), our simple framework which introduces a question generator to the default reasoning pipeline. We first show that good stepping stone questions exist and are transferrable, meaning that good questions can be generated, and they substantially help LLMs of various capabilities in solving the target tasks. We next frame stepping stone generation as a post-training task and show that we can fine-tune LLMs to generate more useful stepping stones by SFT and RL on synthetic data.

• The paper introduces ARQ, a framework that integrates a question generator and a solver to create stepping stones that enhance LLM reasoning.
• It demonstrates that high-quality, intermediate questions can improve performance on math benchmarks by up to 13% when generated by capable models.
• The study shows that beneficial stepping stones transfer across diverse solvers and that post-training significantly refines question generation abilities.

Enhancing LLM Reasoning via Stepping Stone Generation: An Analysis of ARQ

Motivation and Framework

Traditional post-training and test-time scaffolds for LLM reasoning—such as Chain-of-Thought, majority voting, and iterative refinement—primarily focus on solving and verifying the given query. However, the capacity to pose productive intermediate questions, or "stepping stones," is underexplored, despite its pivotal role in human cognition and scientific problem-solving. The paper introduces the ARQ (Asking the Right Questions) framework, which brings a question-generation component into the problem-solving pipeline, formalizing the generation of intermediate queries that are relevant, easier, and potentially insight-bearing with respect to the target problem. The ARQ method consists of two modules: a question generator $\phi$ and a solver $\pi$. Given a task $x$, ARQ produces a stepping stone $z \sim \phi(x)$, solves it to obtain $y_z \sim \pi(z)$, and then solves $x$ using the context $(z, y_z)$, i.e., $y \sim \pi(x; z, y_z)$. The approach is applied both as an inference-time scaffold and as a fine-tuning objective for LLMs.

Empirical Evaluation: Inference-Time ARQ

The ARQ framework is empirically studied on challenging math reasoning benchmarks: AIME 2024, AIME 2025, and BeyondAIME. Using variants of GPT-120B with differing reasoning effort, ARQ's performance is compared to Solver-only baselines and other prompting-based decompositions such as Analogical and Least-to-Most. Results indicate that with modern reasoning LLMs, standard prompting-based decompositions degrade performance, while ARQ only provides benefit when the stone generator is a highly capable reasoning model. Less capable models fail to generate helpful stepping stones, highlighting that the ability to ask useful intermediate questions is non-trivial and tightly coupled to model reasoning skill.

Figure 1: Performance comparison shows ARQ with strong stone generator improves beyond Solver-only only on very challenging tasks; prior prompting-based decompositions hurt performance.

Stepping Stone Quality and Variance

A granular analysis of individual stepping stones reveals considerable variance in their utility. While the best stones can increase the solver's success rate by up to 13% (ARQ high), some stones are actively detrimental, reducing the average improvement. The ability to generate highly beneficial stones is positively correlated with the reasoning capacity of the generator. This variance is not replicated with randomly generated stones, confirming that performance gains are due to genuinely insightful stepping stone questions rather than increased context or maximization over rollouts.

Figure 2: Score distribution of stones on AIME 2024 exposes high variance; only the best stones provide significant gains.

Transferability and Universality

The paper demonstrates the transferability of good stepping stones across disparate solvers. Stepping stones selected by one solver (reference) yield comparable improvements when applied to other solvers (target) with different size and reasoning capacity. This universality indicates that the benefits of well-chosen intermediate questions are not narrowly overfit to specific solver biases but represent a broadly useful strategy for curriculum generation.

Figure 3: Transferability experiments show best stones help diverse solvers equally well, regardless of selection source.

Synthetic Data and Post-Training

Stepping stone generation is framed as a post-training task. The quality of a generated stone is formalized as the expected reward for solving the target problem conditioned on the stone, averaged over solver rollouts. Synthetic datasets are constructed via frontier models, and LLMs are fine-tuned with SFT and DPO using high-quality stones. Post-training dramatically improves question-asking capabilities: both strong generalist and reasoning base models are substantially enhanced in their ability to generate useful stepping stones, with performance increases of 2.9–3.1% over Solver-only baselines even with modest dataset sizes.

Figure 3: Post-training improves stepping stone generation for all solvers, making LLMs better at producing helpful intermediate queries.

Multi-stone ARQ and Curriculum Construction

ARQ is extended to allow for multiple stepping stones generated either sequentially or recursively. Sequential stone generation, in particular, significantly improves both the best-case and average performance, facilitating a test-time curriculum analogous to human incremental problem-solving. Gains in sequential ARQ extend even to models that were post-trained only on single stone generation, evidencing strong generalization. Recursive ARQ does not reliably provide further improvement.

Figure 4: Sequential multi-stone generation yields continuous improvement; recursive approach is less effective.

Theoretical and Practical Implications

The results establish stepping stone generation as an impactful yet underutilized dimension of LLM reasoning. The ARQ framework enables models to construct intermediate curricula, bridging the gap between naive decomposition and humanlike incremental problem-solving. The universality of helpful stones across solver architectures and the demonstrable improvement through targeted post-training expose stepping stone generation as a highly promising direction for future research. Scaling the approach to larger datasets, other domains (e.g., coding and agentic environments), and integrating it with online RL could catalyze advances in autonomous curriculum design for AI.

Conclusion

The ARQ framework systematically introduces stepping stone question generation as a core capability for reasoning LLMs. Empirical evidence supports the existence and transferability of beneficial intermediate questions; ARQ leads to significant improvements on difficult reasoning benchmarks, especially when the generator is appropriately post-trained using synthetic data. The research suggests that systematic training on question-generation tasks, as well as multi-stone curriculum construction, can further boost LLMs' ability to tackle complex reasoning problems. Future exploration should extend ARQ's principles to broader domains and investigate online reinforcement learning strategies for adaptive question-asking.