TAC: Truth Anchoring for LLM Uncertainty

• Truth Anchoring (TAC) is a post-hoc calibration method that transforms raw LLM uncertainty scores into probabilities reflecting factual correctness.
• It learns a mapping function, fθ(s), using a small calibration set with correctness labels to align heuristic model scores with empirical truth indicators.
• Empirical results show TAC significantly reduces Expected Calibration Error and improves AUROC on fact-QA benchmarks, enhancing factual reliability.

Truth AnChoring (TAC) is a post-hoc calibration method for LLM uncertainty estimation that maps raw uncertainty scores to truth-aligned scores, with the output interpreted as an estimated probability of correctness. It is introduced in "Towards Reliable Truth-Aligned Uncertainty Estimation in LLMs" (Srey et al., 1 Apr 2026) to address a central failure mode of conventional uncertainty estimation (UE): many UE metrics originate from model behaviour rather than being explicitly grounded in the factual correctness of LLM outputs. The method is motivated by the observation that such metrics can become non-discriminative precisely in low-information regimes, where locally fluent but globally incorrect responses may receive low uncertainty. TAC therefore reframes uncertainty calibration as the learning of a mapping from a raw score $s(X,Y)\in\mathbb{R}$ to a calibrated confidence $f_\theta(s)\in[0,1]$ using held-out correctness labels.

1. Problem Setting and Motivation

The immediate setting for TAC is factual reliability in LLM generation. Given a user query $X$ and an LLM response $Y$, conventional systems often compute a scalar uncertainty score such as token-level entropy, sequence log-probability, or perplexity, and then use that score as a proxy for factual correctness. The motivating claim of TAC is that this proxy relationship is unstable: these scores may correlate with truth in some configurations, but can fail when the response is locally coherent while globally false (Srey et al., 1 Apr 2026).

The underlying concern is not merely miscalibration in the narrow probabilistic sense, but what the paper formalises as proxy failure. In this formulation, common UE metrics reflect local continuation stability in the model rather than global factual consistency of the answer. This means that the score may remain low for both correct and incorrect outputs, especially when hallucinated responses are fluent. A plausible implication is that reliability mechanisms built directly on raw UE scores inherit the structural limitations of the proxy itself rather than the truth criterion they are intended to approximate.

The paper positions TAC as a practical remedy for this mismatch. Rather than replacing an existing UE metric, TAC calibrates it post hoc by learning a truth-aligned transformation from score space to correctness probability. This suggests a separation between two tasks that are often conflated: producing a heuristic model-behaviour score and estimating factual correctness.

2. Formalisation of Proxy Failure

The paper defines a query $X\in\mathcal{X}$, a response $Y\in\mathcal{Y}$, and a gold-truth function $g^\star:\mathcal{X}\times\mathcal{Y}\to\{0,1\}$. The correctness label is

$C=\mathbb{1}[g^\star(X,Y)=1].$

A raw uncertainty score is then written as $s(X,Y)\in\mathbb{R}$. One example given is predictive entropy:

$$S(X,Y) = - \sum_t p(y_t|x,y_{<t}) \log p(y_t|x,y_{<t}),$$

though sequence log-probability and perplexity are also explicitly included in the same role (Srey et al., 1 Apr 2026).

The paper’s central theoretical claim is that the discriminative utility of a score is controlled by its mutual information with correctness. Proposition 1 states that

$f_\theta(s)\in[0,1]$0

so if $f_\theta(s)\in[0,1]$1, corresponding to a low-information regime, then $f_\theta(s)\in[0,1]$2, which is no better than chance. Proposition 2 constructs explicit distributions where predictive entropy $f_\theta(s)\in[0,1]$3 can be made arbitrarily small. In the paper’s interpretation, conventional uncertainty measures can therefore fail as truth proxies precisely when global structure matters and when robust detection of hallucination is most needed.

This formalisation clarifies an important conceptual distinction. Poor performance of a UE metric need not indicate a failure of threshold selection or insufficient calibration data; it may instead reflect that the metric itself carries little information about factual correctness. TAC is designed against this background: it does not assume that raw uncertainty is already truth-aligned, only that some recoverable signal may exist.

3. TAC Formulation and Calibration Mapping

TAC introduces a calibration mapping, described as the “anchor,”

$f_\theta(s)\in[0,1]$4

parameterized by $f_\theta(s)\in[0,1]$5, whose output is interpreted as the model’s estimated probability of correctness (Srey et al., 1 Apr 2026). The objective is binary cross-entropy, explicitly described as a strictly proper scoring rule:

$f_\theta(s)\in[0,1]$6

with

$f_\theta(s)\in[0,1]$7

The functional role of TAC is therefore not to modify generation, retrieve evidence, or alter the raw uncertainty estimator. It learns a one-dimensional score-to-confidence transformation from labelled data. In principle, the paper allows any monotonic mapping from raw score to calibrated confidence. The main experiments use a small multilayer perceptron (MLP) that takes the one-dimensional input $f_\theta(s)\in[0,1]$8 and passes it through three hidden ReLU layers with hidden size $f_\theta(s)\in[0,1]$9, followed by a scalar output head and final sigmoid. The paper also lists, though does not use in the main experiments, parametric monotonic transforms such as piecewise-linear calibrators or a two-parameter sigmoid,

$X$0

with $X$1 learned (Srey et al., 1 Apr 2026).

The significance of this formulation is methodological. TAC treats factual confidence as an estimand learned from correctness supervision rather than as a direct reading of internal model behaviour. This suggests that the paper’s notion of “truth anchoring” is epistemic rather than generative: the method aligns a score with correctness labels after generation has already occurred.

4. Training Protocol and Inference Procedure

TAC is trained on examples $X$2 obtained by running the LLM on a held-out calibration set and annotating each generated response with its correctness label. The paper states that in low-resource regimes, few-shot calibration uses as few as $X$3 gold labels, and that in noisy settings up to $X$4 of labels may be corrupted, for example by a weak automatic verifier, with only minor performance degradation (Srey et al., 1 Apr 2026).

The primary optimization target is

$X$5

The paper also defines an optional pairwise ranking regularizer to directly improve AUC:

$X$6

where $X$7 is the pre-sigmoid logit for example $X$8, and $X$9 is the set of positive-negative pairs. The final loss is

$Y$0

with $Y$1 typically set to $Y$2. Training uses Adam with learning rate $Y$3 and early stopping on validation AUC (Srey et al., 1 Apr 2026).

The paper’s pseudocode characterises TAC as a post-hoc calibration pipeline:

1. For each calibration query $Y$4, generate response $Y$5 via the LLM, compute raw uncertainty score $Y$6, and obtain correctness label $Y$7.
2. Fit $Y$8 by minimizing

$Y$9

1. At inference on a new $X\in\mathcal{X}$0, compute raw score $X\in\mathcal{X}$1 and output calibrated confidence $X\in\mathcal{X}$2.
2. Optionally, treat $X\in\mathcal{X}$3 as an uncertainty measure for flagging hallucinations.

This protocol is deliberately lightweight. It requires no retraining of the underlying LLM and no change to the generation mechanism itself. A plausible implication is that TAC is compatible with a wide range of pre-existing uncertainty scorers so long as they produce a scalar raw score.

5. Empirical Results and Observed Behaviour

The reported experiments span five LLMs—Qwen-3-4B, Llama-3.2-3B, Llama-3.1-8B, Ministral-3-8B, and Gemma-2-9B—and three fact-QA benchmarks: TriviaQA, SciQ, and PopQA (Srey et al., 1 Apr 2026). The paper reports gains on both calibration and, often, discrimination.

For Expected Calibration Error (ECE), the paper gives the following example for Ministral-8B.

Benchmark	Vanilla ECE	TAC ECE
TriviaQA	$X\in\mathcal{X}$4	$X\in\mathcal{X}$5
SciQ	$X\in\mathcal{X}$6	$X\in\mathcal{X}$7
PopQA	$X\in\mathcal{X}$8	$X\in\mathcal{X}$9

For AUROC on the same model, the reported values are:

Benchmark	Vanilla AUROC	TAC AUROC
TriviaQA	$Y\in\mathcal{Y}$0	$Y\in\mathcal{Y}$1
SciQ	$Y\in\mathcal{Y}$2	$Y\in\mathcal{Y}$3
PopQA	$Y\in\mathcal{Y}$4	$Y\in\mathcal{Y}$5

The paper further states that using as few as $Y\in\mathcal{Y}$6 gold labels or $Y\in\mathcal{Y}$7 corrupted labels yields $Y\in\mathcal{Y}$8 drop in $Y\in\mathcal{Y}$9ECE and barely any drop in $g^\star:\mathcal{X}\times\mathcal{Y}\to\{0,1\}$0AUROC. Under cross-dataset transfer, for example training on TriviaQA and testing on SciQ or PopQA, TAC still reduces ECE by $g^\star:\mathcal{X}\times\mathcal{Y}\to\{0,1\}$1--$g^\star:\mathcal{X}\times\mathcal{Y}\to\{0,1\}$2 points on held-out tasks and outperforms stronger baselines like CUE that require BERT-based features (Srey et al., 1 Apr 2026). Reliability diagrams are reported to show that raw scores can be severely over- or under-confident, whereas TAC produces near $g^\star:\mathcal{X}\times\mathcal{Y}\to\{0,1\}$3 calibration curves.

These findings support a nuanced interpretation. TAC’s strongest and most consistent effect is on calibration quality, as measured by ECE. Improvements in discrimination, as measured by AUROC, are often observed but are not guaranteed by construction. This is consistent with the paper’s theoretical framing: calibration can align score values with empirical correctness probabilities when informative structure exists, but cannot create discriminative signal where the raw score carries essentially none.

6. Scope, Limitations, and Conceptual Distinctions

The paper identifies TAC as applicable to any downstream scenario requiring a calibrated confidence in an LLM’s factuality, including automated medical advice, chatbot safety filters, and selective verification. It is described as particularly important in low-information or open-domain settings where proxies fail (Srey et al., 1 Apr 2026).

At the same time, the paper makes several limitations explicit. TAC cannot create discriminative power out of zero-information raw scores. If $g^\star:\mathcal{X}\times\mathcal{Y}\to\{0,1\}$4, then even the optimal $g^\star:\mathcal{X}\times\mathcal{Y}\to\{0,1\}$5 yields chance-level ranking. TAC also requires a small calibration set with ground-truth or verifier labels, although the paper states that only tens of examples suffice. The empirical focus is fact-recall QA; extensions to multi-step reasoning or creative open-ended generation are left for investigation.

The paper also outlines future directions: inter-score anchoring that jointly calibrates multiple heterogeneous uncertainty scores such as entropy and consistency; task-adaptive calibrators that generalize across tasks or models by meta-learning on diverse calibration sets; larger models and more complex architectures for $g^\star:\mathcal{X}\times\mathcal{Y}\to\{0,1\}$6, including embeddings or intermediate LLM activations; and theoretical analysis of when pairwise ranking losses help or hurt calibration-versus-discrimination trade-offs (Srey et al., 1 Apr 2026).

A common misconception addressed implicitly by the TAC framework is that a low raw uncertainty score is already a direct indicator of truth uncertainty. The paper argues against this equivalence. Another potential source of confusion is terminological: TAC concerns post-hoc calibration of uncertainty scores against factual correctness, whereas "Resonant Context Anchoring" is a distinct inference-time intervention that amplifies value-vector magnitudes in self-attention to improve contextual faithfulness (Zhao et al., 1 Jun 2026). The two methods share the language of “anchoring,” but they operate at different stages and solve different problems.

In this sense, TAC occupies a specific methodological niche. It is neither a truth verifier nor a decoding-time control mechanism. It is a calibration layer that attempts to align heuristic uncertainty signals with empirical correctness, under the explicit recognition that heuristic proxies can fail.