Momentum Uncertainty-Guided Reasoning for LLM Inference

• Momentum Uncertainty-guided Reasoning (MUR) is a dynamic, training-free controller that allocates extra compute only to uncertain reasoning steps.
• It uses a recency-weighted uncertainty metric to trigger test-time scaling, reducing overthinking by focusing compute on fragile steps.
• Empirical evaluations on Qwen3 models show over 50% compute savings and up to 3.37% accuracy improvements compared to uniform per-step scaling.

Momentum Uncertainty-guided Reasoning (MUR) is a training-free inference controller for LLMs that adaptively allocates test-time scaling to selected reasoning steps by tracking and aggregating stepwise uncertainty over time (Yan et al., 20 Jul 2025). It was introduced to address the inefficiency of uniform test-time scaling, which can improve reasoning quality but often induces overthinking by spending extra compute on trivial as well as difficult steps. In MUR, the central control signal is not a one-step confidence score but a recency-weighted “momentum uncertainty” that summarizes the recent uncertainty trajectory of the reasoning path. The acronym is overloaded in the literature: in semi-supervised learning, “MUR” denotes Maximum Uncertainty Regularization, not Momentum Uncertainty-guided Reasoning (Do et al., 2020).

1. Problem formulation and conceptual scope

MUR is situated in the setting of test-time scaling (TTS) for reasoning-intensive LLM inference. The target problem is not answer generation in isolation, but the allocation of extra inference-time compute across a multi-step reasoning trajectory. The underlying claim is that reasoning paths are heterogeneous: some steps are straightforward and can be emitted directly, whereas others are fragile, uncertain, or error-prone and therefore deserve more deliberation (Yan et al., 20 Jul 2025).

Within this framing, MUR is not itself a new reasoning algorithm such as Best-of-$N$, critique, or long-thought generation. Instead, it is a controller that decides when to invoke an expensive TTS method $Q$. The paper explicitly positions MUR as orthogonal to existing TTS methods, including Guided Search, LLM as a Critic, $\phi$-Decoding, and Thinking Mode. A plausible implication is that MUR is best understood as a meta-inference mechanism for compute allocation rather than as a replacement for the underlying reasoning procedure.

The motivating criticism is directed at two baselines. First, per-step scaling applies extra compute to every reasoning step and therefore wastes budget on easy steps. Second, naive uncertainty control is treated as insufficient: raw step uncertainty is described as noisy and unstable, while simple averages over past uncertainty are described as biased because they weight old and recent steps equally even when the reasoning state is changing (Yan et al., 20 Jul 2025). MUR’s core contribution is to turn uncertainty from a point estimate into a trajectory-sensitive control signal.

2. Formal mechanism

The model generates reasoning steps autoregressively as

$a_t \sim p_\theta(\cdot \mid x, \mathbf{a}_{<t}),$

where $x$ is the input question and $\mathbf{a}_{<t}$ is the previously generated reasoning. A test-time scaling method $Q$ produces an optimized step

$\hat{a}_t \sim Q(\cdot \mid x, \mathbf{a}_{<t}).$

MUR introduces a binary detector $D(t)$ that selects between the raw and scaled step: $$\hat{a}_t= \begin{cases} Q(\cdot \mid x,\mathbf{a}_{<t}) &, D(t)=\mathrm{True} \ a_t &, D(t)=\mathrm{False}. \end{cases}$$

The primitive uncertainty signal is step-level average negative log-likelihood. If step $Q$0 contains $Q$1 tokens, the step uncertainty is

$Q$2

This is a likelihood-based quantity aggregated from token probabilities to the step level. That choice distinguishes MUR from a number of other uncertainty-guided methods that use predictive entropy directly.

MUR then defines momentum uncertainty by the recursion

$Q$3

with $Q$4. The paper expands this into the exponential moving average form

$Q$5

which makes the recency weighting explicit: recent steps receive larger weights than distant ones.

The decision rule is the paper’s $Q$6-control mechanism. At step $Q$7, scaling is triggered when

$Q$8

which is equivalent to

$Q$9

Because $\phi$0 in practice, the threshold lies above the historical baseline $\phi$1. The detector therefore ignores small fluctuations and reacts only when current uncertainty rises sufficiently above the momentum-smoothed trajectory. The paper states that smaller $\phi$2 yields fewer scaled steps and lower token usage, whereas larger $\phi$3 yields more scaled steps and higher token usage; at the limit, $\phi$4 is equivalent to Per-Step Scale (Yan et al., 20 Jul 2025).

The implementation details refine this controller. The first reasoning step is not scaled, because there is no valid prior momentum uncertainty. After a step is scaled, the momentum update uses the uncertainty of the optimized step actually emitted, not the discarded raw candidate. Budget control is implicit rather than separately optimized: MUR does not learn a halting policy, but regulates where extra compute is spent via $\phi$5.

3. Theoretical analysis

The paper provides several formal claims intended to justify the use of momentum smoothing for uncertainty control (Yan et al., 20 Jul 2025). The first claim is structural: $\phi$6 is an exponentially weighted sum of past step uncertainties. This is interpreted as a form of discounted credit assignment over reasoning history, in which early noise fades and recent behavior dominates.

The second claim concerns stability. Under the stylized noise model

$\phi$7

the variance of momentum uncertainty is derived as

$\phi$8

Since the multiplicative factor is less than $\phi$9 for $a_t \sim p_\theta(\cdot \mid x, \mathbf{a}_{<t}),$0, the paper concludes that $a_t \sim p_\theta(\cdot \mid x, \mathbf{a}_{<t}),$1. In the paper’s terminology, this is the basis for claiming that momentum uncertainty is more stable than raw step uncertainty.

The third claim concerns bias and convergence. Assuming a drifting latent signal $a_t \sim p_\theta(\cdot \mid x, \mathbf{a}_{<t}),$2 with exponential convergence to a stationary value $a_t \sim p_\theta(\cdot \mid x, \mathbf{a}_{<t}),$3, the paper derives an exponential bound

$a_t \sim p_\theta(\cdot \mid x, \mathbf{a}_{<t}),$4

and concludes that the expected bias tends to zero as $a_t \sim p_\theta(\cdot \mid x, \mathbf{a}_{<t}),$5. The stated interpretation is that momentum uncertainty improves the variance-bias tradeoff relative to simpler estimators.

The paper also compares MUR to the simple-average baseline

$a_t \sim p_\theta(\cdot \mid x, \mathbf{a}_{<t}),$6

Its argument is that equal weighting makes the average less appropriate for non-stationary reasoning trajectories, because stale uncertainty is retained with the same importance as recent uncertainty. This comparison is central to the paper’s framing: MUR is presented not merely as smoothing, but as recency-sensitive smoothing.

Finally, the appendix derives a confidence-style tail bound for the scaling rule by rewriting the detector in terms of the threshold

$a_t \sim p_\theta(\cdot \mid x, \mathbf{a}_{<t}),$7

The paper interprets the resulting bound as a probabilistic justification for dynamic scaling. The assumptions behind these results are explicitly stylized, including bounded or sub-Gaussian uncertainty variables, weak dependence, and smoothly drifting latent uncertainty. This suggests that the theoretical section is intended primarily as support for the design intuition rather than as a full statistical characterization of LLM decoding.

4. Experimental evidence

MUR is evaluated on recent Qwen3 models—Qwen3-1.7B, Qwen3-4B, and Qwen3-8B—across four reasoning benchmarks: MATH-500, AIME24, AIME25, and GPQA-diamond (Yan et al., 20 Jul 2025). The main TTS settings are Guided Search, LLM as a Critic, $a_t \sim p_\theta(\cdot \mid x, \mathbf{a}_{<t}),$8-Decoding, and a Thinking Mode switch. The global inference defaults are temperature $a_t \sim p_\theta(\cdot \mid x, \mathbf{a}_{<t}),$9, $x$0, and $x$1, unless otherwise specified. In the thinking-switch setup, the model uses non-thinking mode by default and switches to thinking mode when MUR fires; the maximum thinking length per step is limited to $x$2.

The headline result is that MUR reduces computation by over 50% on average while improving accuracy by 0.62–3.37% relative to Per-Step Scale (Yan et al., 20 Jul 2025). The strongest gains appear in settings where per-step scaling is expensive and broadly over-applied. For example, on Qwen3-1.7B with Guided Search, MUR improves average accuracy from $x$3 to $x$4 while reducing tokens from $x$5 to $x$6. On Qwen3-8B with $x$7-Decoding, it improves average accuracy from $x$8 to $x$9 while reducing tokens from $\mathbf{a}_{<t}$0 to $\mathbf{a}_{<t}$1. In thinking-mode control, the gains are smaller in efficiency but remain positive in accuracy: on Qwen3-4B, MUR improves average accuracy from $\mathbf{a}_{<t}$2 to $\mathbf{a}_{<t}$3 while reducing tokens from $\mathbf{a}_{<t}$4 to $\mathbf{a}_{<t}$5.

A notable empirical finding is that MUR scales only a minority of reasoning steps. Across settings, the backbone generates about 4.23–6.24 steps on average, while MUR scales about 0.41–1.15 steps per query. Benchmark-level scaled-step percentages are reported as 8.38% on MATH-500, 13.19% on AIME24, 17.83% on AIME25, and 13.75% on GPQA-diamond. This supports the paper’s claim that MUR concentrates compute on harder subproblems rather than applying a uniform budget.

The ablations reinforce the detector’s selectivity claim. Randomly scaling the same number of steps underperforms Per-Step Scale by 1.02% on average, whereas MUR outperforms both random scaling and Per-Step Scale. MUR also exceeds the Avg uncertainty baseline by about 2.20% on average and SMART by about 1.88% on average. The paper interprets these comparisons as evidence that neither single-step uncertainty nor uniform averaging is sufficient for stepwise compute allocation.

The $\mathbf{a}_{<t}$6 ablation functions as a compute-accuracy dial. Increasing $\mathbf{a}_{<t}$7 leads to more scaled steps, more tokens, and generally higher accuracy until overthinking begins to dominate. The $\mathbf{a}_{<t}$8 ablation reports weaker performance at very low momentum, notably $\mathbf{a}_{<t}$9, where the baseline uncertainty changes too quickly. This suggests that MUR’s practical behavior depends on preserving enough temporal inertia to distinguish transient fluctuations from persistent uncertainty.

5. Relation to adjacent uncertainty-guided reasoning methods

MUR belongs to a broader family of methods that use uncertainty to control inference, but its mechanism is unusually explicit about temporal aggregation. In “Partial Reasoning in LLMs: Search and Refinement Guided by Uncertainty,” PREGU monitors token-level entropy during autoregressive generation, halts at the first token whose entropy crosses a threshold, and launches localized latent-space refinement from the partial reasoning prefix. The paper explicitly states that it does not use temporal smoothing, cumulative uncertainty, uncertainty momentum, or persistence-based triggers (Luz et al., 17 Jan 2026). Relative to PREGU, MUR replaces a first-crossing trigger with a recency-weighted historical baseline.

In multi-hop QA, “Memory-Aware and Uncertainty-Guided Retrieval for Multi-Hop Question Answering” introduces MIND, which uses token-level entropy and an attention-derived influence statistic to trigger retrieval and combines retrieval with a memory of retrieved facts. The same summary explicitly notes that MIND has persistent evidence memory but without explicit momentum over uncertainty signals (Ji et al., 29 Mar 2025). MUR’s state variable is thus qualitatively different: it stores a trajectory of uncertainty, not a cache of retrieved evidence.

Several 2025–2026 reasoning systems use uncertainty at other granularities. “Adaptive Multi-Expert Reasoning via Difficulty-Aware Routing and Uncertainty-Guided Aggregation” applies a router-side uncertainty score over easy-versus-hard problem classification to allocate sample diversity, but its uncertainty is estimated once from the problem text rather than updated over the reasoning path (Ehab et al., 11 Apr 2026). “AdaThink-Med: Medical Adaptive Thinking with Uncertainty-Guided Length Calibration” uses entropy-based uncertainty and correctness during reinforcement learning to calibrate reasoning length, but its only momentum-like mechanism is EMA smoothing of batch statistics rather than stepwise momentum over a live reasoning trajectory (Rui et al., 29 Sep 2025). “When to Think and When to Look: Uncertainty-Guided Lookback” introduces a training-free controller for LVLMs that uses grounding-sensitive local triggers and visual lookback prompts; the paper characterizes its own online policy as local and reactive rather than momentum-based (Bi et al., 19 Nov 2025). “LLMs for Game Theory: Entropy-Guided In-Context Learning and Adaptive CoT Reasoning” uses token-level entropy to adapt retrieval and branching in Tic-Tac-Toe, and explicitly suggests future dynamic thresholds based on previous-step entropy distributions, making it a clear precursor to more stateful control (Banfi et al., 15 Jan 2026).

A separate terminological clarification is essential. The 2020 semi-supervised learning paper “Semi-Supervised Learning with Variational Bayesian Inference and Maximum Uncertainty Regularization” defines MUR as Maximum Uncertainty Regularization, an input-space consistency loss driven by predictive entropy in an $Q$0-bounded neighborhood (Do et al., 2020). That method is unrelated to the LLM inference controller introduced in 2025 except for acronym overlap. In current LLM reasoning literature, MUR refers specifically to Momentum Uncertainty-guided Reasoning (Yan et al., 20 Jul 2025).

6. Limitations, interpretation, and open directions

The paper’s main practical strength is simplicity: MUR requires no fine-tuning, no reinforcement learning, no auxiliary classifier, and only lightweight overhead—computing token probabilities, averaging step NLL, maintaining one scalar $Q$1, and evaluating one threshold inequality (Yan et al., 20 Jul 2025). This makes it straightforward to insert into existing TTS pipelines. At the same time, the method inherits several limitations from its design.

First, MUR depends on the quality of token-probability-based uncertainty. The paper uses average step NLL as a proxy for uncertainty, but does not present an explicit calibration study. This suggests that poor calibration or systematic misestimation of token likelihoods could affect the detector. Second, the empirical scope is concentrated on four reasoning benchmarks and the Qwen3 model family. The results are broad within that setting, but external generalization is not established in the same detail.

Third, the controller is binary: it decides whether to scale a step, not how much to scale it. The conclusion explicitly identifies this as future work. A plausible implication is that MUR’s next natural extension is from binary gating to graded budget allocation, in which uncertainty would modulate branch width, search depth, or critique budget continuously rather than as an on-off decision. Fourth, the theory relies on stylized assumptions—weak dependence, bounded or sub-Gaussian variables, and smoothly drifting latent uncertainty—that serve as analytical support rather than direct empirical validation.

More broadly, MUR’s place in the literature is that of a trajectory-aware uncertainty controller. Other recent systems show that uncertainty can trigger retrieval, local refinement, grounding interventions, or adaptive reasoning length, but they generally remain reactive, problem-level, or training-time in orientation (Luz et al., 17 Jan 2026, Ji et al., 29 Mar 2025, Ehab et al., 11 Apr 2026, Rui et al., 29 Sep 2025, Bi et al., 19 Nov 2025, Banfi et al., 15 Jan 2026). MUR’s distinctive claim is that inference should react not to isolated uncertainty values, but to deviations from a smoothed uncertainty history. This suggests a broader research direction in which uncertainty is treated as a dynamic state variable of reasoning itself.