How Uncertainty Estimation Scales with Sampling in Reasoning Models

Abstract: Uncertainty estimation is critical for deploying reasoning LLMs, yet remains poorly understood under extended chain-of-thought reasoning. We study parallel sampling as a fully black-box approach using verbalized confidence and self-consistency. Across three reasoning models and 17 tasks spanning mathematics, STEM, and humanities, we characterize how these signals scale. Both self-consistency and verbalized confidence scale in reasoning models, but self-consistency exhibits lower initial discrimination and lags behind verbalized confidence under moderate sampling. Most uncertainty gains, however, arise from signal combination: with just two samples, a hybrid estimator improves AUROC by up to $+12$ on average and already outperforms either signal alone even when scaled to much larger budgets, after which returns diminish. These effects are domain-dependent: in mathematics, the native domain of RLVR-style post-training, reasoning models achieve higher uncertainty quality and exhibit both stronger complementarity and faster scaling than in STEM or humanities.

• The paper shows that verbalized confidence (VC) at low sample counts (e.g., 71.3 AUROC at K=1) scales effectively as more samples are drawn, especially in math.
• The study reveals that fusing VC with self-consistency (SC) into a hybrid estimator (SCVC) yields higher AUROC improvements at minimal sampling, outperforming individual methods.
• The paper finds that uncertainty estimation performance is domain dependent, with mathematics benefiting most from combined uncertainty signals compared to STEM and humanities.

Uncertainty Estimation in Reasoning LLMs: Sample Scaling and Signal Complementarity

Introduction

The paper "How Uncertainty Estimation Scales with Sampling in Reasoning Models" (2603.19118) addresses the critical problem of quantifying uncertainty in reasoning LLMs (RLMs) through black-box methods. Focusing on both verbalized confidence (VC) and self-consistency (SC), the work systematically analyzes their scaling properties under parallel sampling regimes using three state-of-the-art open-source RLMs across diverse domains: mathematics, STEM, and humanities. This is the first comprehensive large-scale empirical study delineating how introspective (VC), agreement-based (SC), and hybrid (SCVC) uncertainty signals behave as the number of test-time samples increases.

Problem Formulation and Methodology

Uncertainty estimation for LLMs has been widely studied in the context of non-reasoning models, but RLMs exhibit structurally different inference dynamics: each sample inherently involves an extended chain-of-thought (CoT) trace, potentially exposing richer uncertainty information without requiring deep sampling. Standard uncertainty tools—self-consistency (majority or agreement statistics across samples) and introspective confidence elicitation (verbalized numeric self-assessment)—interact nontrivially after multi-step deliberative reasoning, and prior results from standard LLMs cannot be immediately transferred due to these architectural and training differences.

The work investigates three approaches:

• Verbalized Confidence (VC): Explicit numeric introspection ($c_i \in [0,1]$) attached as a self-reported confidence estimate for each solving chain, aggregated across those samples agreeing with the majority.
• Self-Consistency (SC): The proportion of samples aligning with the majority answer ($\hat{a}$) as a surrogate for posterior answer uncertainty.
• Hybrid Combination (SCVC): A convex combination of SC and VC (default $\lambda=0.5$), probing for complementarity or redundancy in their uncertainty content.

Macro-averaged AUROC is employed as the primary metric, as it is robust to calibration pathologies often observed in verbalized confidence, especially when absolute probability values are not well-aligned with empirical correctness rates.

Main Empirical Findings

Sample Scaling of Uncertainty Signals

The study observes that both VC and SC improve as the number of samples $K$ increases, but VC is consistently more discriminative in the low-sample regime—a single VC estimate is already a strong predictor of answer correctness, especially in mathematics-focused benchmarks.

• In mathematics, VC@1 achieves 71.3 AUROC, scaling to 81.4 at $K=8$ (approximately +10 AUROC).
• In STEM and humanities, scaling is more modest (+4.6 and +4.1 AUROC from $K=1$ to $K=8$).
• SC starts below VC by a significant margin and, although it scales up, does not surpass VC within the sensible sampling budget (up to $K=8$).

These sample efficiency disparities indicate that extended intra-sample CoT effectively exposes substantial uncertainty information, diminishing additional gains of SC under modest sampling.

Figure 1: AUROC vs. computational cost for extended (high-effort) and shallow (low-effort) chain-of-thought sampling in gpt-oss-20b variants.

Signal Complementarity: SC + VC

The most notable result is the substantial benefit of fusing VC and SC, realized with minimal additional sampling overhead. The hybrid SCVC estimator at $K=2$ already outperforms what either pure VC or SC can achieve even at $K=8$—the gains are pronounced:

• In mathematics, SCVC@2 improves AUROC by +12.9 points over VC@1, and more than 4 points over either VC@8 or SC@8.
• In STEM and humanities, the analogous improvements are +6.4, +1.8, and +2.3 AUROC, respectively.

Most domain tasks exhibit strongly diminishing returns past $K=2$ for hybrid estimation. The effect is most prominent in mathematics, where both the scale-up and complementarity are largest and sustained gains persist to higher $K$.

Domain Specificity

The advantage of both VC scaling and signal complementarity is domain dependent, being largest in mathematics—a domain where RLVR (reinforcement learning with verifiable rewards) post-training concentrates—while STEM and humanities saturate far earlier in $K$.

Figure 2: Kendall’s tau rank correlation between VC and SC as a function of $K$, demonstrating complementarity is domain- and depth-sensitive, especially pronounced in mathematics.

Mathematics tasks benefit from both larger and more persistent augmentation via SCVC (e.g., +2.7 AUROC from $K=2 \to 5$, +1.5 for $K=5 \to 8$), while other domains show minimal improvement beyond $K=2$.

Instructional Robustness and Sampling Protocol

The hybrid method’s gain is robust to the weighting parameter $\lambda$, so long as both signals contribute ($0 < \lambda < 1$), and does not depend sensitively on the form of the confidence elicitation prompt (though vanilla/simple variants perform best). Elicitation-based VC (in situ self-report, often with "epistemic" or "verification" prompt priming) is more effective and less costly than judgment-based alternatives (judge models evaluating reasoning traces).

Figure 3: Overview of uncertainty elicitation prompt families (vanilla, verification, epistemic), illustrating prompt design space for VC.

Implications

Black-box uncertainty quantification in RLMs, under realistic computational constraints, should not over-invest in deep sampling for either VC or SC alone. Instead, practitioners should deploy shallow parallel sampling ($K=2$) and fuse both self-consistency and introspective signals, obtaining uncertainty discrimination at far lower cost. This is especially relevant in domains with heavy CoT (such as mathematics) and where per-sample computational cost is high.

These findings reveal nontrivial interaction structure between intra-sample reasoning (exposing model epistemic state) and inter-sample agreement (marginal posterior over answers). When both signals are simultaneously available, their joint exploitation is necessary to extract the maximum attainable uncertainty discrimination from a fixed compute budget.

From a theoretical standpoint, the strong complementarity at low $K$ and its rapid decay as $K$ increases is governed by the fact that each sample in RLMs already integrates over many intermediate hypotheses; only under modestly diverse sampling do the two signals track genuinely distinct information about uncertainty. As more samples are drawn, VC and SC become increasingly correlated.

Domain dependence—highest complementarity in mathematics—is consistent with RLVR model post-training focusing on verifiability and solution checking in this space, whereas less-structured domains (humanities, STEM) are not as optimally tuned for uncertainty emission or exploitation.

Broader Impact and Future Directions

Current deployment protocols for LLM-based systems in high-stakes settings (e.g., medical, legal, educational) will benefit from these results by enabling more efficient and reliable risk estimation for selective prediction and human-in-the-loop configurations. These results provide actionable sampling guidance for black-box uncertainty estimation, crucial when cost per sample is prohibitive.

Future avenues should focus on expanding the diversity and coverage of reasoning models, assessing signal combination in models trained for non-mathematical RLVR tasks, and exploring integration of alternative uncertainty aggregation strategies beyond simple linear convex combination. Further, theoretical models connecting intra-sample reasoning trace variability and inter-sample coherence could elucidate the underlying mechanism of uncertainty quantification in reasoning architectures.

Conclusion

This paper establishes that in reasoning LMs, both verbalized confidence and self-consistency scale with parallel sampling; however, their hybridization yields superior uncertainty discrimination at minimal sampling budgets, especially in mathematics. This suggests that high-quality black-box uncertainty estimation in RLMs is best achieved not by costlier deep sampling, but by combining multiple weakly-correlated signals arising from model introspection and sample-level agreement. This finding is central for the practical deployment and trust calibration of advanced reasoning models.