Sequential Sampling in Large Reasoning Models

Updated 12 April 2026

The paper introduces sequential sampling as a method that generates candidate solutions in an ordered chain, where each sample conditions on previous outputs for refined reasoning.
Techniques like Power-SMC, incremental sampling without replacement, and adaptive early stopping are employed to enhance efficiency, diversity, and reduction in computational load.
Empirical studies demonstrate that sequential sampling can improve accuracy (e.g., pass@1 metrics) and token efficiency compared to parallel sampling methods in structured reasoning tasks.

Sequential sampling in large reasoning models encompasses a family of methods that generate candidate solutions, reasoning trajectories, or answer distributions via a chain of dependent samples, where each iteration may condition on all or part of the previous computational history. This paradigm stands in contrast to parallel (ancestral) sampling, in which multiple reasoning chains are generated independently. Modern sequential approaches address critical challenges in LLM-based reasoning—including distribution sharpening, efficiency under fixed compute budgets, exploration-diversity trade-offs, and adaptive test-time scaling—while posing unique methodological and theoretical questions regarding optimality, diversity collapse, and resource allocation.

1. Core Principles and Mathematical Formalization

Sequential sampling is fundamentally characterized by the generation of candidate solutions in an ordered sequence, where each output may depend on prior outputs and/or reasoning paths. Abstractly, for an underlying conditional policy $\pi_\theta$ representing the model and a static prompt $x$ , the $n$ -th candidate $(z_n, y_n)$ is drawn via

$z_n \sim \pi_\theta(z \mid x, \text{history}_{n-1}), \qquad y_n \sim \pi_\theta(y \mid x, \text{history}_{n-1}, z_n)$

where $\text{history}_{n-1}$ is an (explicitly or implicitly defined) function of previous sampled outputs $(z_1, y_1, \ldots, z_{n-1}, y_{n-1})$ (Gu et al., 7 Apr 2026, Troshin et al., 22 Sep 2025, Kang et al., 17 Oct 2025). Typical instantiations include:

Auto-regressive sequential sampling: Retains all past answers in the prompt context.
Markov sequential sampling: Conditions only on the previous answer or feedback, not the entire chain.
Iterative or conditioned candidate generation: Each drawn output is elicited by prompting for “a new, different solution,” explicitly listing all previous outputs (Troshin et al., 22 Sep 2025).

The joint distribution over generated outputs $y_{1:k}$ under fully auto-regressive conditioning is

$p(y_{1:k} \mid x) = \prod_{i=1}^{k} p(y_i \mid x, y_{<i})$

By contrast, parallel sampling draws all outputs independently: $p_{\text{parallel}}(y_{1:k} \mid x) = \prod_{i=1}^k p(y_i \mid x)$

2. Algorithmic Architectures and Notable Sequential Schemes

A diverse set of sequential sampling methodologies exists for large LLMs, each tailored to particular inference objectives, efficiency regimes, or exploration desiderata.

Sequence-Level Power Sampling via Power-SMC

Power-SMC targets the sharpened sequence-level power distribution $x$ 0 for $x$ 1 (Azizi et al., 10 Feb 2026). This distribution concentrates mass on high-likelihood reasoning trajectories at the joint sequence level, notably differing from per-token temperature scaling. Power-SMC applies a Sequential Monte Carlo (SMC) framework:

Generates a batch of $x$ 2 particles in parallel, each advanced token-wise by a prefix-only proposal $x$ 3.
Assigns and corrects importance weights incrementally, resampling when the Effective Sample Size (ESS) falls below a threshold.
Optionally employs exponent-bridging ( $x$ 4-ramping) schedules to stabilize particle weights.
Returns a sample according to the final normalized weights.

Power-SMC efficiently matches or exceeds Metropolis–Hastings (MH) power sampling in accuracy while reducing inference latency by an order of magnitude (Azizi et al., 10 Feb 2026).

Incremental Sampling Without Replacement

To ensure diversity in output sets, incremental sampling without replacement operates by drawing outputs one by one such that no output is repeated. The algorithm uses a trie-based UniqueRandomizer to maintain probability mass over as-yet-unsampled continuations, with each new sample drawn conditionally to exclude previously visited traces. Computational and memory requirements scale only with the number of unique prefixes, making this method practical for exponentially large output spaces (Shi et al., 2020).

Early Stopping and Adaptive Sequencing

Statistically principled early stopping via sequential hypothesis testing—such as renewal-process statistics or maxwise conformal testing—has been introduced to halt decoding when uncertainty or ill-posedness is detected, reducing unnecessary overlength reasoning without sacrificing correctness. Both parametric and nonparametric approaches provide formal guarantees on false alarm rates and power (Xie et al., 15 Feb 2026). Additionally, sequential probability ratio tests (SPRT) have been successfully applied for adaptive sampling in self-consistency protocols, yielding dramatic savings in token budget with no loss in aggregate accuracy (Lee et al., 22 Mar 2025).

Sequential Test-Time Scaling

Test-time scaling via recursive thought prompting—forcing the model to extend or revise its reasoning chain—is subject to instability and diminishing returns at large numbers of induced iterations. The Min-Seek algorithm stabilizes this process by retaining only the shortest “reconstruction cycle” and the initial prompt+first-thought in the attention cache, achieving linear instead of cubic scaling in compute and enabling robust, unbounded-depth reasoning (Metel et al., 14 Jan 2026).

Exploration-Driven Sequential Sampling in RL Contexts

Sequential sampling has been adapted to reinforcement learning (RL) fine-tuning of LLM policies, specifically to mitigate entropy collapse. SESA (Sequential Sampling for Exploration) generates method sketches sequentially, each conditioned on prior sketches to guarantee diversity, before expanding them in parallel to full solutions for policy optimization. This results in substantially higher coverage of reward-supporting modes and prevents the policy collapse characteristic of standard parallel RL sampling regimes (Kang et al., 17 Oct 2025).

3. Performance Comparison, Trade-offs, and Empirical Findings

Empirical studies comparing sequential and parallel sampling in large reasoning models reveal nuanced trade-offs contingent on task structure, feedback quality, and diversity requirements.

Exploration vs. Exploitation: Sequential chains tend to under-explore, producing highly similar or even verbatim duplicates as conditioning accumulates, unless explicit diversity-promoting mechanisms or oracle feedback are integrated. Parallel sampling reliably maintains higher candidate diversity but lacks the self-refinement affordances of sequential chaining (Gu et al., 7 Apr 2026, Troshin et al., 22 Sep 2025, Karan et al., 16 Oct 2025).
Aggregation Operator Influence: Introducing identical aggregation (e.g., majority vote or best-of- $x$ 5) to both sequential and parallel paradigms does not close accuracy or diversity gaps, indicating that the central limitation in sequential mode is not aggregation per se (Gu et al., 7 Apr 2026).
Context Length and Resource Scaling: The expansion of context in auto-regressive sequential sampling is not a primary source of performance degradation, except for models with inherently limited long-context capabilities. Memory and compute requirements can be mitigated by storing only active reasoning cycles, as in Min-Seek (Metel et al., 14 Jan 2026).
Task Dependence: Backtracking-based sequential search outperforms parallel best-of- $x$ 6 sampling in tasks with high search-tree depth (e.g., Sudoku), but underperforms in shallow-tree tasks (e.g., CountDown), as quadratic growth in attention cost and prescribed search-trace bias can dominate (Qin et al., 9 Apr 2025).
Sample Efficiency and Mode Recovery: Methods such as hint-marginalization (iterative refinement via answer-conditioned hints) improve the efficiency of identifying the most probable answer, outperforming static self-consistency or progressive hint prompting on benchmarks such as GSM8K and SVAMP (Pal et al., 2024).
Training-Free vs. RL-Tuned Regimes: Sequential “power” sampling based on MCMC or SMC at sequence level can match or even exceed RL-finetuned models in pass@1 accuracy on MATH500 and HumanEval, while also preserving pass@k diversity—RL approaches often exhibit diversity collapse (Karan et al., 16 Oct 2025, Azizi et al., 10 Feb 2026).

4. Diversity, Duplication, and Exploration Mechanisms

A principal challenge in sequential sampling is the phenomenon of diversity collapse, where conditioning on prior outputs leads to repeated or minimally varied candidate solutions. Multiple investigations confirm:

Lexical and Solution Diversity: Iterative/sequential generation with explicit re-prompting for “new, different” solutions can induce substantial gains in lexical and computational flow diversity over pure parallel sampling, at a modest cost in mean accuracy (typically <2 percentage points for GSM8K) (Troshin et al., 22 Sep 2025).
Duplication Avoidance: Incremental without-replacement samplers guarantee unique outputs and can exploit early stopping to reduce resource usage once a solution is found (Shi et al., 2020).
Entropy Preservation in RL: SESA’s auto-conditioned sequential sketching preserves entropy and avoids convergence to a single reward-dominant mode that plagues parallel RL rollouts (Kang et al., 17 Oct 2025).
Role of Oracle Feedback: Incorporating high-quality, externally verifiable feedback within the sequential chain partially restores diversity, suggesting that model-internal self-critique alone is insufficient (Gu et al., 7 Apr 2026).

Strategies to enhance exploration within sequential samplers include raising sampling temperature, explicit diversity rewards, randomized restarts, and hybrid schemes that interleave sequential and parallel draws.

5. Computational Efficiency and Implementation Considerations

Efficient implementation of sequential sampling in LLMs is critical due to the potential for context bloat, quadratic cost scaling, and memory exhaustion.

Method	Compute Complexity	Memory Use	Practical Bottleneck
Parallel (ancestral)	$x$ 7	$x$ 8 per sample	None (if ample parallel compute)
Sequential (autoregressive)	$x$ 9 or higher	$n$ 0	Context length, attention cost
Min-Seek	$n$ 1 (linear in cycles)	$n$ 2 (small, fixed)	Requires KV cache modification
Power-SMC	$n$ 3– $n$ 4 baseline	$n$ 5	Linear in particle count $n$ 6
UniqueRandomizer (WOR)	$n$ 7 per sample	$n$ 8	Trie size for long/deep outputs

Batched and GPU-Friendly Decoding: Power-SMC and most sequential schemes leverage GPU batch parallelism to advance multiple chains or particles simultaneously (Azizi et al., 10 Feb 2026). Min-Seek further reduces wall-clock time by discarding extraneous cached states (Metel et al., 14 Jan 2026).

Scaling Beyond Maximum Context: Min-Seek’s key innovation is the dynamic re-encoding of positional keys in the cache, allowing unbounded-depth sequential reasoning without exceeding hardware limits (Metel et al., 14 Jan 2026).

Statistical Control: Sequential probability ratio testing in ConSol introduces adaptive early stopping, providing rigorous Type I/Type II error bounds while minimizing average sample requirements (Lee et al., 22 Mar 2025). Renewal-process and conformal approaches provide robust guarantees for uncertainty-aware early stopping (Xie et al., 15 Feb 2026).

6. Open Challenges and Future Directions

Research to date reveals both strengths and persistent limitations of sequential sampling in the context of large reasoning models:

Closing the Exploration Gap: Despite greater representational capacity, standard sequential samplers under-explore relative to parallel counterparts unless diversity-promoting signals or oracle-based feedback are integrated. Development of hybrid or dynamically adaptive schemes represents a promising direction (Gu et al., 7 Apr 2026, Kang et al., 17 Oct 2025).
Scalable Power Sampling: Further work on particle-efficient SMC variants, stratified resampling, and hybrid SMC/rollout architectures could lead to more efficient targeting of complex sequence-level objectives (Azizi et al., 10 Feb 2026).
Stable Test-Time Scaling: Techniques such as Min-Seek demonstrate that judicious cache management and cycle selection can stabilize very long inference chains, but broader applicability to multimodal or instruction-following models remains an open area (Metel et al., 14 Jan 2026).
Task-Specific Adaptations: The empirical superiority of sequential vs. parallel sampling is dictated by the structure of the underlying reasoning space (e.g., search-tree depth), the quality of feedback and training signals, and the tolerance for redundancy vs. thoroughness (Qin et al., 9 Apr 2025).
Early Stopping/Navigating Ill-Posedness: Statistically rigorous, model-agnostic early-stopping methods are maturing, but integration with more expressive uncertainty signals and policy-level interventions has not been fully explored (Xie et al., 15 Feb 2026).

7. Representative Experimental Insights

Large-scale empirical comparisons anchor several high-impact findings in the literature:

On MATH500 (Qwen2.5-7B): Baseline pass@1=49.8%, Power-SMC=71.4% at only $n$ 9 baseline latency, far outperforming low-temperature token-level decoding and matching RL-finetuned models (Azizi et al., 10 Feb 2026).
Self-consistency with sequential probability ratio testing (ConSol) attains comparable or higher accuracy relative to fixed-sample baselines (e.g., 70% $(z_n, y_n)$ 080% on AIME24), with up to 86% fewer tokens needed (Lee et al., 22 Mar 2025).
SESA sequential exploration yields +0.25–0.42 absolute success rate improvements over base RL models in agent benchmarks, with up to 211% relative improvement (Kang et al., 17 Oct 2025).
Min-Seek enables stable, linear-cost sequential scaling: budget-forcing is 44% slower, with Min-Seek accuracy stabilizing for $(z_n, y_n)$ 1 induced thoughts (Metel et al., 14 Jan 2026).
Sequential backtracking is advantageous when search spaces are deep (e.g., Sudoku) but less so for shallow spaces (e.g., CountDown), especially under compute budget constraints (Qin et al., 9 Apr 2025).

Sequential sampling remains a central topic in the test-time optimization of large reasoning models, balancing the competing demands of search diversity, inference fidelity, computational efficiency, and real-time practical constraints. The trajectory of recent work suggests a convergence toward methods that exploit explicit structure—distribution sharpening, batched SMC, hint-marginalization—alongside adaptive, statistically rigorous controls to maximize reasoning performance across increasingly complex reasoning domains.