Truth Scores: Metrics for AI Oversight

• Truth scores are quantitative measures assigning truthfulness by defining a truth condition, a scoring unit, and an estimation rule.
• They are applied at multiple model levels, from attention-head validation accuracy to calibrated probability estimators in output-level evaluations.
• Across domains, truth scores facilitate improved oversight in risk scoring, segmentation in medical imaging, and crowdsourced misinformation assessments.

Searching arXiv for papers on “truth scores” and closely related formulations across LLMs, oversight, and evaluation. arXiv search query: "all:truth scores OR ti:TruthfulQA OR ti:TruthProbe OR ti:Truth Forest OR ti:Truth Anchoring OR ti:TruthTorchLM" Truth scores are quantitative measures that assign truthfulness, correctness, or truth-aligned reliability to a scored object. In recent arXiv literature, the scored object varies widely: an individual attention head can receive a validation-accuracy score for context-grounded truthful reasoning, a reported type in scored elicitation can be evaluated through an incentive ratio governing dominant-strategy truthful reporting, and claims or sources in a truth-discovery graph can be assigned final strengths under gradual semantics (Choi et al., 14 Jun 2026, Lovén, 5 May 2026, Yin et al., 2024). A plausible implication is that “truth score” is not a single standardized metric, but a family of measurement schemes whose meaning is fixed by the truth condition of the task.

1. Conceptual scope and semantic regimes

Across the literature, a truth score is defined by three ingredients: a truth condition, a unit of scoring, and an estimation rule. In context-grounded language modeling, the truth condition is whether an answer is faithfully grounded in provided text, images, or both; the unit may be an attention head or a full model response; and the estimation rule may be a linear probe, a judge model, or a calibrated uncertainty estimator. In scored oversight, the truth condition is honest reporting of a private type, the unit is the report itself, and the estimation rule is the expected score induced by a proper scoring rule. In truth-discovery and argumentation, the truth condition is encoded by graph semantics, the unit is a claim or source, and the estimation rule is the fixed point of an update operator. In learned evaluators for prediction quality, the truth condition is agreement with ground truth, the unit is a predicted output, and the estimation rule is a regression model trained to approximate an external metric such as Dice (Choi et al., 14 Jun 2026, Lovén, 5 May 2026, Senbi et al., 2024).

This variation produces several recurring interpretations. One is truthfulness as factual or contextual correctness, common in LLM evaluation. Another is truthfulness as incentive compatibility, where a score quantifies how much honest reporting dominates misreporting. A third is truth/trust as gradual strength, where claims and sources receive continuous values rather than binary labels. A fourth is truth as calibrated correctness probability, where the score is intended to approximate $P(\text{correct} \mid \cdot)$. This suggests that comparison across papers requires attention to what is being scored and to what notion of truth the score is aligned.

2. Internal and head-level truth scores in LLMs

A particularly explicit formulation appears in “The Truth Stays in the Family,” where a Truth Score is defined per attention head as the validation accuracy of a binary linear probe trained on that head’s output at the final answer token to distinguish truthful from hallucinated answers. For head $(\ell,h)$ with activation $x^{(\ell,h)} \in \mathbb{R}^{d_h}$, the probe is

$$p_\theta(x^{(\ell,h)}) = \sigma(\theta^\top x^{(\ell,h)}),$$

and the score is

$$\text{TruthScore}(\ell,h)=\text{validation accuracy of the probe},$$

averaged over 5-fold cross-validation. The paper studies HaluEval, PhD, and RLHF-V, and reports that these head-level scores are strongly preserved within model families such as Vicuna-, Qwen2.5-, LLaMA2-, and Mistral-based lineages. Within-family correlations between base LLMs and descendants are reported as approximately $0.77$–$0.98$, cross-dataset within-lineage correlations remain $0.51$–$0.64$, and an unrelated model such as Mistral-7B shows near-zero correlation of approximately $0.04$–$(\ell,h)$0 with Vicuna-7B. This inheritance is consistent with attention-head weight preservation: within-family Frobenius-norm drift is approximately $(\ell,h)$1, whereas cross-family drift is approximately $(\ell,h)$2–$(\ell,h)$3; the top-20 truthful heads in LLaVA-1.5 are mostly in middle and deep layers, with 80% in layers 10–31. The same paper introduces TruthProbe, a training-free soft-gating rule

$(\ell,h)$4

applied to head outputs at inference time, and reports improvements on HaluEval, POPE, and CHAIR, with base-LLM Truth Scores transferring effectively to fine-tuned LLM and MLLM descendants (Choi et al., 14 Jun 2026).

Related work treats truth scores as geometric or probabilistic properties of internal states rather than as head-ranking statistics. “Truth Forest” uses sigmoid probes $(\ell,h)$5 as probabilities that a hidden state is in a truthful state, and also treats the signed projection $(\ell,h)$6 as a directional measure of truthfulness. It extends single-axis probing by training multiple orthogonal probes per head, combines them with exponential decay weighting, and uses Random Peek to sample hidden states from varying answer positions rather than only the final token. The learned axes are then used for inference-time intervention. On TruthfulQA, the paper reports that Truth Forest improves Llama-2-7B from 40.8% to 74.5%, and that orthogonal probes capture complementary truth-related features in well-defined clusters (Chen et al., 2023).

3. Output-level truthfulness, uncertainty, and calibration

At the output level, TruthfulQA established a benchmark view in which truthfulness is measured on 817 questions spanning 38 categories, with answers scored on a scalar in $(\ell,h)$7 and thresholded at $(\ell,h)$8 for binary reporting. The benchmark defines an answer as truthful iff it avoids asserting a false statement, explicitly separating truthfulness from informativeness. Under this protocol, the best model in the original evaluation was truthful on 58% of questions, while human performance was 94% (Lin et al., 2021). Subsequent benchmark work broadens the same idea. “Truth Knows No Language” creates a professionally translated, fully parallel extension of TruthfulQA in Basque, Catalan, Galician, and Spanish, and reports that Judge-LLM correlates more closely with human judgments than MC2, that informativeness plays a critical role in truthfulness assessment, and that machine translation yields no statistically significant difference relative to professional translation for these languages. For instruct models, the average Judge-LLM truth scores are reported as 76.5 in English, 72.2 in Spanish, 69.1 in Catalan, 67.1 in Galician, and 55.1 in Basque (Figueras et al., 13 Feb 2025). “TruthEval” shifts emphasis from benchmark questions to 885 curated statements over Fact, Conspiracy, Controversy, Misconception, Stereotype, and Fiction, and treats prompt-stability as part of reliability by comparing responses across prompt variants P0–P4 (Khatun et al., 2024).

A parallel line of work treats truth scores as calibrated correctness probabilities derived from uncertainty signals. “Between the Layers Lies the Truth” converts each layer activation at a task-relevant token into a probability distribution

$(\ell,h)$9

builds a layer–layer signature map with

$x^{(\ell,h)} \in \mathbb{R}^{d_h}$0

and trains a small classifier to output

$x^{(\ell,h)} \in \mathbb{R}^{d_h}$1

with uncertainty defined as $x^{(\ell,h)} \in \mathbb{R}^{d_h}$2. The method matches probing in-distribution and outperforms probing under cross-dataset transfer and 4-bit quantization (Badash et al., 17 Mar 2026). “Towards Reliable Truth-Aligned Uncertainty Estimation in LLMs” formalizes proxy failure for heuristic uncertainty metrics and proposes Truth AnChoring (TAC), a post-hoc calibration method that learns

$x^{(\ell,h)} \in \mathbb{R}^{d_h}$3

using binary cross-entropy and an optional ranking term. The paper proves the bound

$x^{(\ell,h)} \in \mathbb{R}^{d_h}$4

and reports that TAC substantially reduces ECE even with few-shot or noisy supervision (Srey et al., 1 Apr 2026). “TruthTorchLM” systematizes this space in software form, providing over 30 truthfulness prediction methods, a unified interface for generation, evaluation, calibration, and long-form truthfulness prediction, and support for both HuggingFace and LiteLLM (Yaldiz et al., 10 Jul 2025).

4. Scored oversight, elicitation, and risk scoring

In scored oversight, truth scores are not primarily about factual content but about incentive-compatible honest reporting. “Honest Reporting in Scored Oversight” studies a heterogeneous scored elicitation mechanism in which an agent of private type $x^{(\ell,h)} \in \mathbb{R}^{d_h}$5 reports to a principal evaluated by a power-$x^{(\ell,h)} \in \mathbb{R}^{d_h}$6 pseudospherical scoring rule with $x^{(\ell,h)} \in \mathbb{R}^{d_h}$7. The paper defines the incentive ratio

$x^{(\ell,h)} \in \mathbb{R}^{d_h}$8

and derives the exact gain formula

$x^{(\ell,h)} \in \mathbb{R}^{d_h}$9

which implies dominant-strategy incentive compatibility for all $$p_\theta(x^{(\ell,h)}) = \sigma(\theta^\top x^{(\ell,h)}),$$0 because $$p_\theta(x^{(\ell,h)}) = \sigma(\theta^\top x^{(\ell,h)}),$$1. It then defines the True-KL$$p_\theta(x^{(\ell,h)}) = \sigma(\theta^\top x^{(\ell,h)}),$$2 property as

$$p_\theta(x^{(\ell,h)}) = \sigma(\theta^\top x^{(\ell,h)}),$$3

for all $$p_\theta(x^{(\ell,h)}) = \sigma(\theta^\top x^{(\ell,h)}),$$4, $$p_\theta(x^{(\ell,h)}) = \sigma(\theta^\top x^{(\ell,h)}),$$5, and $$p_\theta(x^{(\ell,h)}) = \sigma(\theta^\top x^{(\ell,h)}),$$6, proving that for these dimensions the score is not merely truth-inducing but gives a uniform multiplicative margin against misreporting, with

$$p_\theta(x^{(\ell,h)}) = \sigma(\theta^\top x^{(\ell,h)}),$$7

The proof uses the substitution $$p_\theta(x^{(\ell,h)}) = \sigma(\theta^\top x^{(\ell,h)}),$$8, Prékopa’s theorem on log-concavity preservation, dimension-specific analytic arguments, and certified numerics; for $$p_\theta(x^{(\ell,h)}) = \sigma(\theta^\top x^{(\ell,h)}),$$9 the paper locates a critical threshold $$\text{TruthScore}(\ell,h)=\text{validation accuracy of the probe},$$0, while for $$\text{TruthScore}(\ell,h)=\text{validation accuracy of the probe},$$1 True-KL$$\text{TruthScore}(\ell,h)=\text{validation accuracy of the probe},$$2 fails (Lovén, 5 May 2026).

LLM-based risk scoring generalizes the same probabilistic viewpoint to unrealizable prediction tasks. “Evaluating LLMs as risk scores” defines a score function

$$\text{TruthScore}(\ell,h)=\text{validation accuracy of the probe},$$3

and operationalizes it through multiple-choice prompting by normalizing the next-token probabilities of options “A” and “B”. On ACS-based tasks, the paper finds that zero-shot risk scores from multiple-choice question answering have high predictive signal but are widely miscalibrated: base models consistently overestimate outcome uncertainty, while instruction-tuned models underestimate uncertainty and produce over-confident risk scores, with instruction-tuning polarizing answer distributions regardless of true underlying data uncertainty. A separate experiment using verbalized chat-style risk queries yields substantially improved calibration across instruction-tuned models (Cruz et al., 2024).

5. Truth and trust scores in structured reasoning and crowdsourced assessment

In structured argumentation, truth scores can be the end state of an explicit semantics rather than the output of a probe or calibrator. “Applying Attribution Explanations in Truth-Discovery Quantitative Bipolar Argumentation Frameworks” instantiates Truth-Discovery QBAFs, where arguments are sources and claims, contradictory claims attack one another, sources and their claims support one another, and base scores are $$\text{TruthScore}(\ell,h)=\text{validation accuracy of the probe},$$4 for sources and $$\text{TruthScore}(\ell,h)=\text{validation accuracy of the probe},$$5 for claims. Under Quadratic Energy semantics, each argument receives a final strength

$$\text{TruthScore}(\ell,h)=\text{validation accuracy of the probe},$$6

with updates driven by supporter-minus-attacker energy. In this setting, $$\text{TruthScore}(\ell,h)=\text{validation accuracy of the probe},$$7 is the trustworthiness score of source $$\text{TruthScore}(\ell,h)=\text{validation accuracy of the probe},$$8, while $$\text{TruthScore}(\ell,h)=\text{validation accuracy of the probe},$$9 is the truth score of claim $0.77$0. The same paper studies Argument Attribution Explanations and Relation Attribution Explanations, showing how individual nodes and edges contribute to the final truth score of a target claim even in cyclic graphs (Yin et al., 2024).

Crowdsourced misinformation assessment extends the notion further by treating truth scores as aggregated social judgments. “The Many Dimensions of Truthfulness” proposes a multidimensional representation with seven dimensions—Correctness, Neutrality, Comprehensibility, Precision, Completeness, Speaker’s Trustworthiness, and Informativeness—plus an Overall Truthfulness judgment, all collected on a 5-point Likert scale and aggregated by arithmetic mean. The paper reports that the crowdsourced assessments are reliable when compared to an expert-provided gold standard, that the proposed dimensions capture independent pieces of information, and that Random Forest models trained on these assessments can predict expert labels with accuracies of 0.556 on PolitiFact 6 levels, 0.667 on ABC 3 levels, and 0.518 on ABC’s original fine-grained verdicts (Soprano et al., 2021). A related study, “Can The Crowd Identify Misinformation Objectively?”, compares 3-level, 6-level, and 100-level truthfulness scales, finds low internal agreement across all scales, shows that mean aggregation outperforms median and mode for matching expert labels, and reports that crowd truth scores become markedly more reliable when collapsed into coarse binary or ternary bins. It also finds that higher CRT performance and some political indicators are associated with higher discernment (Roitero et al., 2020).

6. Application domains beyond language generation and recurring limitations

Outside text generation, truth scores often appear as learned proxies for ground-truth quality. “Towards Ground-truth-free Evaluation of Any Segmentation in Medical Images” defines EvanySeg as a regression model

$0.77$1

where the target quality is typically Dice score. The score is learned from image–segmentation pairs and then used at test time without access to ground-truth masks. The paper reports that ViT backbones outperform ResNet-101, that the model can identify poorly segmented samples, benchmark models on unlabeled sets, and select among candidate segmentations; in model selection, ViT-b EvanySeg chooses the truly best segmenter in approximately 90.8% of cases on TG3K and approximately 77.0% on FLARE21, yielding mean Dice close to oracle selection (Senbi et al., 2024). In a distinct biomedical setting, “Enhancing Machine Learning Performance with Continuous In-Session Ground Truth Scores” treats pain intensity as a continuous in-session ground-truth signal sampled at 2 Hz, aligns it with EDA features over 10-second windows, and shows that MLP and Random Forest models trained with in-session scores achieve macro-averaged geometric mean scores of 75.9% and 78.3%, compared with 70.3% and 74.6% when trained on post-session VAS scores (Faremi et al., 2023).

A further variant treats truth scores as interval-valued error proxies. “Interval Neural Networks: Uncertainty Scores” learns lower and upper bounds around the output of a pretrained network, uses interval width as an uncertainty score, and exploits asymmetry for directional information. The paper reports approximately 89% coverage on the 1DDeconv task, $0.77$2 pixel coverage on CT, and direction accuracy reaching approximately 72% for high direction thresholds, while PWCC on CT is $0.77$3, slightly above MC Dropout (Oala et al., 2020). Across these disparate domains, recurring limitations are consistent: scores depend on the chosen architecture and data regime; transfer across unrelated families or strong distribution shift is fragile; noisy or subjective labels can distort calibration; and improved scores do not eliminate hallucinations, prediction error, or the need for human oversight. This suggests that truth scores are best understood as operational instruments for ranking, calibration, abstention, or diagnosis, rather than as direct substitutes for truth itself.