How do LLMs Compute Verbal Confidence

Abstract: Verbal confidence -- prompting LLMs to state their confidence as a number or category -- is widely used to extract uncertainty estimates from black-box models. However, how LLMs internally generate such scores remains unknown. We address two questions: first, when confidence is computed - just-in-time when requested, or automatically during answer generation and cached for later retrieval; and second, what verbal confidence represents - token log-probabilities, or a richer evaluation of answer quality? Focusing on Gemma 3 27B and Qwen 2.5 7B, we provide convergent evidence for cached retrieval. Activation steering, patching, noising, and swap experiments reveal that confidence representations emerge at answer-adjacent positions before appearing at the verbalization site. Attention blocking pinpoints the information flow: confidence is gathered from answer tokens, cached at the first post-answer position, then retrieved for output. Critically, linear probing and variance partitioning reveal that these cached representations explain substantial variance in verbal confidence beyond token log-probabilities, suggesting a richer answer-quality evaluation rather than a simple fluency readout. These findings demonstrate that verbal confidence reflects automatic, sophisticated self-evaluation -- not post-hoc reconstruction -- with implications for understanding metacognition in LLMs and improving calibration.

• The paper demonstrates that LLMs automatically cache confidence during answer generation, supporting a second-order evaluation mechanism.
• It employs activation steering, patching, and noising to pinpoint temporal dynamics at key tokens (PANL and CC) across transformer layers.
• Comparisons with token log-probabilities reveal that PANL and CC contain unique, decodable confidence information, enhancing calibration strategies.

Mechanistic Analysis of Verbal Confidence Computation in LLMs

Problem Statement and Theoretical Motivation

The paper "How do LLMs Compute Verbal Confidence" (2603.17839) investigates the internal computation of verbal confidence in LLMs, focusing on when and how confidence representations arise during inference. Verbal confidence—where models are prompted to state confidence as a categorical or numeric value—is widely used for uncertainty estimation, but its mechanistic underpinnings within transformer architectures remain obscure. The authors contrast two hypotheses: (1) Just-in-time (JIT) computation, where confidence is only computed at the prompt site when explicitly requested, versus (2) Cached retrieval, where confidence is automatically computed and stored during answer generation, independent of whether a rating will later be solicited. The study further assesses whether verbal confidence is reducible to token log-probabilities (first-order accounts) or reflects a richer, second-order answer-quality evaluation.

Figure 1: Main Prompt and schematic illustration of sequential confidence information flow—answer tokens $\rightarrow$ PANL $\rightarrow$ CC.

Experimental Methodology

The authors employ a multi-pronged mechanistic interpretability paradigm using Gemma 3 27B and Qwen 2.5 7B as primary testbeds. Key interventions include activation steering, patching (corrupt-then-restore), noising (mean ablation), activation swapping, and attention blocking, each dissecting the spatial and temporal loci of confidence computation. The focus is on analyzing the representations of the post-answer-newline (PANL) token, the confidence-colon (CC), and relevant control positions under both categorical and numeric prompting regimes. Behavioral metrics include confidence change, logit difference change, and first token change rate; decodability is assessed via linear probing and variance partitioning.

Sequential Emergence and Causal Roles of Confidence Representations

Activation Steering: The authors demonstrate that steering confidence via direct manipulation at the PANL token robustly modulates verbal confidence output, with peak efficacy in layers 21–25. Steering at the CC token is also effective but peaks in later layers (30–35), conforming to serial information flow from answer-adjacent tokens towards final verbalization. Control positions such as PANL+1 and FCC exhibit negligible effects.

Figure 2: Activation steering at PANL and CC shows efficacious confidence modulation, with early layer effects at PANL and later layer effects at CC.

Activation Patching: Degrading all answer token activations via mean ablation reduces confidence output to near floor, but selective patching at PANL or CC partially (PANL) or nearly fully (CC) restores confidence-related metrics. Temporal precedence is evident—PANL patching achieves recovery earlier in the network relative to CC (peak layer 25 vs 30+), reinforcing the cached retrieval hypothesis.

Figure 3: Activation patching at PANL and CC restores confidence outputs after upstream corruption; recovery at PANL precedes CC.

Activation Noising: Mean ablation at PANL or CC partially disrupts confidence reporting, confirming their necessity. Control positions are unaffected.

Figure 4: Activation noising at PANL and CC disrupts verbal confidence output, quantified by logit difference reduction and token change rate.

Activation Swap: Swapping PANL activations between low/high confidence donor and high/low recipient trials induces systematic bidirectional shifts in verbal confidence, beyond generic content disruption observed in same-confidence controls. This causally implicates PANL as a persistent confidence-caching locus.

Figure 5: Cross-confidence activation swaps at PANL alter recipient confidence; same-confidence swaps yield stability, confirming confidence-specific information transfer.

Figure 6: Directional effects in swap experiments peak at layer 26, robust across all behavioral metrics.

Information Decodability and Distinction from Log-Probabilities

Linear probes trained on PANL and CC activations decode both binary correctness (AUROC) and verbal confidence magnitude ($R^2$), with decodability at PANL emerging in earlier layers than all other positions. Variance partitioning shows PANL and CC explain substantial variance in confidence reporting beyond mean log-probabilities ($R^2_{\text{CV}} = 0.084$ for logprobs vs higher for PANL/CC), a contradictory result with respect to the first-order account.

Figure 7: Confidence and correctness information is decodable from PANL and CC at distinct layers; PANL decodability precedes CC and encodes information not captured by token log-probabilities.

Attention Blocking and Information Flow Pathways

Attention blocking experiments using minimal numeric prompts establish the information flow: answer tokens $\rightarrow$ PANL (layers 22–28) $\rightarrow$ CC (layers 30–36). Blocking CC’s attention to PANL induces marked disruption; blocking CC$\rightarrow$Q+A yields minimal effect, invalidating the JIT hypothesis. PANL’s attention to answer tokens is earlier, with input disruption preceding downstream CC effects.

Figure 8: Attention blocking reveals sequential confidence information transfer—early disruptions at PANL's attention to answer tokens, followed by CC's attention to PANL.

Generalization Across Architectures and Prompt Regimes

Replication in Qwen 2.5 7B with categorical prompts and Gemma with numeric prompting recapitulates temporal precedence (PANL $\rightarrow$ CC) and robust intervention effects at both positions. Activation steering, patching, swap, and noising show comparable qualitative patterns, confirming the generality of cached retrieval across model families and formats.

Calibration and Behavioral Analysis

Gemma and Qwen demonstrate reasonable calibration for verbal confidence (ECE: 0.12/0.06; AUROC: 0.71/0.65). Distribution plots confirm utilization across confidence spectra, reinforcing practical viability for uncertainty quantification.

Figure 9: Gemma calibration (ECE = 0.12, AUROC = 0.71) and confidence class distribution reveals model's self-evaluative reliability.

Figure 10: Qwen calibration (ECE = 0.06, AUROC = 0.65) and balanced confidence reporting across classes.

Theoretical and Practical Implications

The findings substantiate a second-order account of confidence computation in LLMs: confidence is computed and cached during answer generation, not simply reconstructed from log-probabilities at the prompt site. This architecture supports latent error detection and sophisticated metacognitive self-evaluation, aligning with analogs in decision neuroscience. From a calibration perspective, exploiting cached confidence representations could enable principled interventions beyond prompt engineering or generic fine-tuning, and enhance reliability under black-box deployment where token-level probabilities are inaccessible.

Conclusion

The paper establishes that LLMs compute verbal confidence via automatic cached retrieval during answer generation, consolidating information at answer-adjacent tokens that is subsequently retrieved at the verbalization site. Confidence representations at PANL and CC contain information not reducible to token-level log-probabilities, supporting richer evaluation mechanisms consistent with second-order theories. Sequential mechanistic tracing using causal interventions and attention blocking elucidates the temporal and spatial flow of confidence information. These results are robust across models and prompt regimes, providing a foundational basis for future research on introspection, metacognitive error detection, and calibration strategies in large-scale generative models.