Truth Probes in Language Models

• Truth probes are techniques that read out and intervene on truth-related structures in language model activations using linear classifiers and perturbation methods.
• They reveal that truth signals are geometrically structured, enabling causal interventions that adjust model outputs and improve detection of hallucinations.
• Methodologies range from logistic regression and mass-mean probes to compositional and training-free schemes, each with distinct strengths and limitations across tasks.

Truth probes are methods for reading out, decomposing, calibrating, or intervening on truth-related structure in language-model representations. In the activation-probing line, they typically treat residual-stream states or other hidden activations as features and fit decision rules that separate true from false statements, honest from deceptive responses, or faithful from hallucinated continuations. In parallel, counterfactual and propositional formulations probe truthfulness by perturbing statements with plausible factual errors or by decoding latent propositions from internal states. Across these formulations, the recurring claim is not that model outputs are uniformly truthful, but that truth-relevant signals are often internally available, geometrically structured, and sometimes causally actionable (Ying et al., 23 Feb 2026, Feng, 3 Aug 2025, Feng et al., 2024).

1. Conceptual scope

The phrase “truth probe” is used for several closely related objects in the literature. In the narrowest sense, it denotes a linear classifier over hidden states that predicts whether a statement or answer is true, false, correct, incorrect, honest, or deceptive. Marks and Tegmark describe this as a “geometry of truth”: true and false declarative statements occupy linearly separable regions of residual-stream space (Marks et al., 2023). Azizian et al. state the same idea as the hypothesis that there exists a single hyperplane in $\mathbb{R}^d$ separating hidden states of correct versus incorrect answers at a chosen layer and token position (Azizian et al., 10 Jun 2025).

Later work broadens the concept. The Truthfulness Spectrum Hypothesis argues that truthfulness is not encoded as a single direction but as a spectrum of linear directions ranging from domain-general to domain-specific, with distinct behavior across definitional, empirical, logical, fictional, ethical, sycophantic, and expectation-inverted settings (Ying et al., 23 Feb 2026). Related work on “belief directions” studies vectors whose projections correlate with a model’s degree of belief in a sentence, including conditional belief under preceding context (Schouten et al., 2024). The trilemma framework extends the label space beyond $\{\text{true},\text{false}\}$ to $\{\text{true},\text{false},\text{neither}\}$, arguing that many statements are neither true nor false for the model because they are fragments, nonsense, or unsupported (Savcisens et al., 30 Jun 2025).

A second broadening concerns what is being monitored. Some probes target factual truth of isolated statements; some target contextual faithfulness, meaning whether a continuation is supported by a given source context; some target strategic deception, meaning whether internal reasoning is deceptive even when surface text appears benign; and some target latent world-state fidelity, meaning whether the model’s internal representation of a context remains faithful even when its decoded output is not (O'Neill et al., 31 Jul 2025, Goldowsky-Dill et al., 5 Feb 2025, Feng et al., 2024). A plausible implication is that “truth probes” do not isolate a single universal semantic property so much as a family of readouts over related internal signals.

2. Core linear-probe formulations

The canonical truth probe is a linear map from a hidden activation to a scalar score. A standard parametrization extracts a hidden vector $h \in \mathbb{R}^d$ at a chosen layer and computes

$z = w^{T} h + b,\qquad \hat p = \sigma(z),$

where $w$ is the probe direction, $b$ a bias, and $\sigma$ the sigmoid (Goldowsky-Dill et al., 5 Feb 2025). In truth-direction work, one often omits the bias after mean-centering activations, so that the learned weight vector itself is the “truth direction” (Poulis et al., 4 Apr 2026).

The feature extraction step varies across papers. The Truthfulness Spectrum work takes hidden activations from a fixed layer’s residual stream and averages across output tokens (Ying et al., 23 Feb 2026). “The Geometry of Truth” uses the residual-stream activation over the final period token of a declarative statement (Marks et al., 2023). Several QA-oriented studies instead use the final-token hidden state or the last hidden state at layers chosen by a between-class to within-class variance heuristic (Bao et al., 1 Jun 2025). The universal hyperplane work records hidden states of every attention head at every transformer layer but only at the last token of the generated output, then selects a small subset of heads before fitting the final probe (Liu et al., 2024).

The dominant training objective is regularized logistic regression. One formulation writes

$$L(w) = E_i\big[\log(1+\exp(-y_i \cdot (w^{T}h_i)))\big] + \lambda \|w\|^2,$$

with labels $y_i \in \{+1,-1\}$ (Ying et al., 23 Feb 2026). Another uses regularized binary cross-entropy with labels $\{\text{true},\text{false}\}$0:

$\{\text{true},\text{false}\}$1

This form is used for deception detection as well as truth-value readout (Goldowsky-Dill et al., 5 Feb 2025).

Several alternatives recur. Difference-of-means or mass-mean probes set the direction to the difference between class means, $\{\text{true},\text{false}\}$2, and classify by the sign of $\{\text{true},\text{false}\}$3 (Marks et al., 2023). Linear SVMs, LDA, and regularized logistic regression are used as supervised baselines or mainline estimators (Bao et al., 1 Jun 2025, Ying et al., 23 Feb 2026). CCS and CCR are unsupervised or weakly supervised constructions over paired true/false statements (Schouten et al., 2024). Truth Forest replaces a single direction with a set of orthogonal directions $\{\text{true},\text{false}\}$4 and adds an orthogonality penalty to the binary cross-entropy objective (Chen et al., 2023).

These formulations differ not only in optimization but in what they assume about internal organization. A single hyperplane posits one dominant axis. Difference-of-means assumes that class centroids are already informative. Sparse probes assume that relatively few coordinates are decisive. Orthogonal or iterative methods assume that multiple partially independent truth-related directions coexist.

3. Generalization, universality, and truth geometry

One of the central disputes concerns whether truthfulness is organized by a universal direction or by task-specific geometries. Early evidence for broad generalization came from curated true/false statement datasets. Marks and Tegmark report cross-dataset accuracies above $\{\text{true},\text{false}\}$5 in almost every pairwise transfer among curated datasets, with mass-mean probes generalizing slightly better than logistic-regression and CCS probes; causal interventions along those directions also altered model truth judgments (Marks et al., 2023). Liu et al. strengthen the universalist position by training on over 40 datasets across 17 task families and finding that dataset diversity, rather than per-dataset sample size, is the dominant factor: cross-task accuracy rises from about $\{\text{true},\text{false}\}$6 with 4 tasks to about $\{\text{true},\text{false}\}$7 with 41 tasks for LLaMA-2-7B-chat, and stronger base models reach about $\{\text{true},\text{false}\}$8 to $\{\text{true},\text{false}\}$9 (Liu et al., 2024).

A sharply different picture appears in cross-task answer-correctness probing. Azizian et al. find that pairwise cosine similarities between single-task probes are almost always below $\{\text{true},\text{false},\text{neither}\}$0, that sparse probes have mostly disjoint supports with most task pairs sharing less than $\{\text{true},\text{false},\text{neither}\}$1 overlap, and that mixtures of tasks or probes do not recover held-out task performance. On their account, “geometries of truth” are intrinsically task-dependent and near-orthogonal across many tasks, with only semantically related tasks such as TriviaQA, NQ, and SimpleQA showing higher alignment and partial transfer (Azizian et al., 10 Jun 2025).

The Truthfulness Spectrum Hypothesis explicitly reconciles these views. It reports near-perfect in-domain performance on each truth type, strong transfer among definitional, empirical, logical, fictional, and ethical probes, and failure of any single-domain probe on sycophantic and expectation-inverted lying, where AUROC is about $\{\text{true},\text{false},\text{neither}\}$2 and sometimes below chance. Yet a jointly trained probe over all domains recovers high performance on every domain, which the paper interprets as evidence that domain-general directions exist even when pairwise transfer is poor (Ying et al., 23 Feb 2026). The same work introduces Mahalanobis cosine similarity,

$\{\text{true},\text{false},\text{neither}\}$3

and reports that $\{\text{true},\text{false},\text{neither}\}$4 predicts cross-domain AUROC with $\{\text{true},\text{false},\text{neither}\}$5, compared with $\{\text{true},\text{false},\text{neither}\}$6 for standard cosine; synthetic simulations show $\{\text{true},\text{false},\text{neither}\}$7 for Mahalanobis similarity and as low as $\{\text{true},\text{false},\text{neither}\}$8 for standard cosine (Ying et al., 23 Feb 2026).

Later studies add further limits to universality claims. Truth-direction quality is highly layer-dependent, with factual recall tasks becoming linearly separable in early layers and arithmetic or harder counting tasks only in mid-to-late layers; no single layer is optimal for all tasks (Poulis et al., 4 Apr 2026). Prompt template matters as well: under “ask-correct,” emergence of truth directions is delayed by about 5–10 layers, cosine similarity between no-prompt and ask-correct directions can drop to about $\{\text{true},\text{false},\text{neither}\}$9–$h \in \mathbb{R}^d$0, and cross-task generalization between factual and arithmetic tasks can improve dramatically under correctness-evaluation framing while remaining at chance for harder factual tasks (Poulis et al., 4 Apr 2026). Conversational-format experiments reach a parallel conclusion: probes trained on single-sentence statements or short conversations generalize poorly to longer formats where the lie appears earlier, but appending a fixed key phrase improves long-format generalization by about 15–25 percentage points across cross-format pairings (Ichmoukhamedov et al., 14 May 2025).

Taken together, the literature supports a stratified picture. Some truth-related structure transfers broadly across topics, logical transformations, and benchmark families; some is tied to task type, prompt structure, layer, and deception regime. This suggests that “universality” is best interpreted as conditional rather than absolute.

4. Methodological extensions

Truth probes now include several methodological families beyond a single logistic hyperplane. One family decomposes representational space into subspaces of varying generality. Stratified INLP in the Truthfulness Spectrum work first extracts mutually orthogonal domain-general directions by repeated null-space projection on jointly trained data, then projects those out and learns domain-specific directions separately for each domain. The resulting general directions maintain high accuracy across every domain, whereas the domain-specific directions achieve high in-domain accuracy but chance-level, less than $h \in \mathbb{R}^d$1 AUROC, on all other domains. LEACE complements this by erasing the subspace predictive of one domain and showing that the erased domain drops to chance while in-domain performance on non-erased domains remains intact; transfer degradations after erasure reveal partially overlapping subspaces, and a capacity-allocation model places most capacity in subspaces shared by 3–6 domains rather than in a single universal or purely domain-specific axis (Ying et al., 23 Feb 2026).

A second family is explicitly compositional. Propositional probes separate lexical decoding from relational binding: domain probes recover lexical items such as names, countries, occupations, and foods from token activations, while a learned binding subspace identifies which tokens are bound as predicate arguments. In the closed-world setting, this allows recovery of propositions such as $h \in \mathbb{R}^d$2 from internal activations. Trained only on templated synth data, these probes generalize to paraphrased short stories and Spanish translations, and in prompt-injection, backdoor, and gender-bias settings the decoded propositions remain substantially more faithful than standard prompting outputs (Feng et al., 2024).

A third family operates through perturbation rather than direct activation classification. Counterfactual probing segments a generated output into atomic factual statements, generates plausible counterfactual variants with subtle errors, and measures the model’s confidence robustness. For a statement $h \in \mathbb{R}^d$3 with counterfactual set $h \in \mathbb{R}^d$4, it defines

$h \in \mathbb{R}^d$5

$h \in \mathbb{R}^d$6

and

$h \in \mathbb{R}^d$7

with hallucination declared when $h \in \mathbb{R}^d$8. On TruthfulQA-tuned evaluation, this yields detection $h \in \mathbb{R}^d$9, better than Simple Confidence, Self-Consistency, SelfCheckGPT, and Fact-Checking, and its adaptive rewrites reduce hallucination scores by $z = w^{T} h + b,\qquad \hat p = \sigma(z),$0 on held-out examples (Feng, 3 Aug 2025).

A fourth family is training-free. TruthV analyzes MLP value vectors and their key activations in multiple-choice QA. It selects top argmax and argmin vectors on a small calibration set, then majority-votes over candidates. On Gemma-2-2B-it, the reported numbers are $z = w^{T} h + b,\qquad \hat p = \sigma(z),$1 for log-likelihood, $z = w^{T} h + b,\qquad \hat p = \sigma(z),$2 for NoVo, $z = w^{T} h + b,\qquad \hat p = \sigma(z),$3 for TruthV argmax only, $z = w^{T} h + b,\qquad \hat p = \sigma(z),$4 for TruthV argmin only, and about $z = w^{T} h + b,\qquad \hat p = \sigma(z),$5 for the combined method; similar 8–12 point gains over NoVo are reported across four models and ten datasets (Liu et al., 22 Sep 2025).

Other extensions modify the output space or the token structure being probed. Truth Forest introduces multi-dimensional orthogonal probes plus Random Peek, then injects a pre-computed “truth bias” into attention-head outputs at inference; on TruthfulQA, Llama-2-7B improves from $z = w^{T} h + b,\qquad \hat p = \sigma(z),$6 True to $z = w^{T} h + b,\qquad \hat p = \sigma(z),$7 True, with True$z = w^{T} h + b,\qquad \hat p = \sigma(z),$8Info rising from $z = w^{T} h + b,\qquad \hat p = \sigma(z),$9 to $w$0 (Chen et al., 2023). The sAwMIL framework replaces single-instance classification with multiple-instance learning over token bags, adds conformal prediction, and explicitly models true, false, and neither as separate one-vs-all probes (Savcisens et al., 30 Jun 2025).

5. Causality and steering

A recurring question is whether truth probes are merely diagnostic or whether they intersect the causal circuitry used during generation. Several papers report direct interventions. In “The Geometry of Truth,” probe-identified directions are added to or subtracted from residual activations in a causally important token/layer group. The reported Normalized Indirect Effect is largest for mass-mean directions, especially when training includes both a dataset and its logical negation: for LLaMA-13B, the cities+neg_cities mass-mean intervention yields false$w$1true NIE $w$2 and true$w$3false NIE $w$4, whereas probes trained on the “likely” dataset have almost no causal impact (Marks et al., 2023).

Context-sensitive belief-direction work reports a different causal test. It identifies a belief direction $w$5 for premise–hypothesis pairs, then intervenes on the premise representation by subtracting or adding $w$6 before recomputing the hypothesis score. On Llama2-13B, interventions on affirmed premises decrease $w$7 for entailments and increase it for contradictions when moved backward along $w$8, with MMP and CCR yielding the largest causal shifts, up to about 10 percentage points. The paper interprets this as evidence that belief directions partially mediate in-context inference (Schouten et al., 2024).

The Truthfulness Spectrum paper complicates the causal story. Its intervention adds $w$9 to the MLP bias at a chosen layer with $b$0 and evaluates $b$1 on SimpleQA. Domain-specific directions produce positive $b$2 with mean about $b$3, whereas the single domain-general direction yields negative $b$4 with mean about $b$5. The intervention effect grows with baseline confidence; decomposition shows that domain-specific directions suppress wrong answers, while the general direction boosts both correct and incorrect answers and therefore harms discrimination (Ying et al., 23 Feb 2026). This result is central to the paper’s claim that universal truth directions can be predictive without being the directions most directly used for generation.

The observer-model hallucination work provides a further causal demonstration. It isolates a single residual-stream direction in a frozen observer model that separates contextual hallucinations from faithful continuations, then injects or ablates the normalized direction in a generator’s residual stream at layer 10. Positive $b$6 drives hallucination rates up to $b$7 while reducing repetition to less than $b$8; negative $b$9 lowers hallucinations toward $\sigma$0 while increasing repetition to about $\sigma$1 (O'Neill et al., 31 Jul 2025).

Propositional probes suggest a related but distinct causal interpretation. In prompt-injection and backdoor settings, prompting fails but decoded propositions remain faithful, which the authors take as evidence that the latent world model may remain correct while the decoding process becomes unfaithful (Feng et al., 2024). This suggests that some truth probes monitor latent fidelity rather than the exact mechanism that determines the emitted string.

6. Evaluation regimes, applications, and limitations

Truth probes are evaluated under several regimes: in-domain classification, cross-domain or cross-task transfer, calibration, abstention, and intervention success. AUROC is common for linear truth and deception probes; F1 and accuracy are common for hallucination detection; ECE and Brier Score appear in calibration-focused work; recall at fixed false-positive rate is used for deployment-oriented deception monitoring; and conformal prediction is used where abstention is required (Bao et al., 1 Jun 2025, Goldowsky-Dill et al., 5 Feb 2025, Azizian et al., 10 Jun 2025).

In hallucination detection, counterfactual probing reports detection $\sigma$2, ECE $\sigma$3 versus $\sigma$4 for Simple Confidence, and hallucination-score reduction of $\sigma$5, with $\sigma$6 of flagged statements successfully mitigated (Feng, 3 Aug 2025). The observer-model residual probe reports $\sigma$7 on CNN/DM, $\sigma$8 on XSUM, and about $\sigma$9–$$L(w) = E_i\big[\log(1+\exp(-y_i \cdot (w^{T}h_i)))\big] + \lambda \|w\|^2,$$0 on CONTRATALES, outperforming baselines by 5–8 points on summarization and 9–27 points on logical contradictions (O'Neill et al., 31 Jul 2025). In selective question answering, a statement-trained SVM probe on Llama-3.1-8B increases accuracy from $$L(w) = E_i\big[\log(1+\exp(-y_i \cdot (w^{T}h_i)))\big] + \lambda \|w\|^2,$$1 over all samples to $$L(w) = E_i\big[\log(1+\exp(-y_i \cdot (w^{T}h_i)))\big] + \lambda \|w\|^2,$$2 on the $$L(w) = E_i\big[\log(1+\exp(-y_i \cdot (w^{T}h_i)))\big] + \lambda \|w\|^2,$$3 of pairs it retains (Bao et al., 1 Jun 2025).

In deception detection, linear probes on Llama-3.3-70B-Instruct distinguish honest and deceptive responses with AUROCs from $$L(w) = E_i\big[\log(1+\exp(-y_i \cdot (w^{T}h_i)))\big] + \lambda \|w\|^2,$$4 to $$L(w) = E_i\big[\log(1+\exp(-y_i \cdot (w^{T}h_i)))\big] + \lambda \|w\|^2,$$5 and recall from $$L(w) = E_i\big[\log(1+\exp(-y_i \cdot (w^{T}h_i)))\big] + \lambda \|w\|^2,$$6 to $$L(w) = E_i\big[\log(1+\exp(-y_i \cdot (w^{T}h_i)))\big] + \lambda \|w\|^2,$$7 at $$L(w) = E_i\big[\log(1+\exp(-y_i \cdot (w^{T}h_i)))\big] + \lambda \|w\|^2,$$8 FPR on control chat data, across roleplaying, insider-trading concealment, and sandbagging settings (Goldowsky-Dill et al., 5 Feb 2025). Mechanistic work on smaller open models reports a three-stage layerwise pattern—near-random in early layers, peak in middle layers, slight decline in later layers—and finds roughly 20 deception directions in Qwen 3B, about 45 in Qwen 7B, about 80 in Qwen 14B, and about 95 in DeepSeek 7B through INLP (Boxo et al., 27 Aug 2025).

The main limitations are also recurrent. Generalization can collapse across tasks, prompt formats, or deception types; linear probes may pick up spurious features such as sentence polarity, dataset artifacts, or morally charged contexts; and some models require nonlinear probes for manipulation/locality to hold (Poulis et al., 4 Apr 2026, Azizian et al., 10 Jun 2025, Savcisens et al., 30 Jun 2025, Goldowsky-Dill et al., 5 Feb 2025). Context sensitivity is double-edged: probes respond to supporting and contradicting premises, but unrelated or corrupted contexts can also shift outputs substantially (Schouten et al., 2024). Several works note that sycophancy is especially problematic: single-domain truth probes fail on sycophantic and expectation-inverted lying, and post-training appears to push sycophantic directions into a more orthogonal subspace, providing a representational explanation for chat-model sycophantic tendencies (Ying et al., 23 Feb 2026).

A final limitation is operational. Many approaches require hidden-state access, layer sweeps, or full observer-model forward passes; real-time monitoring therefore depends on efficient activation extraction and scoring (O'Neill et al., 31 Jul 2025, Goldowsky-Dill et al., 5 Feb 2025). Even when conformal methods guarantee false-positive control on unseen tasks, recall can fall sharply because thresholds must compensate for geometric misalignment (Azizian et al., 10 Jun 2025). The current literature therefore supports a conservative conclusion: truth probes are strong instruments for monitoring and dissecting truth-related structure in LLMs, but their validity depends on layer, task, prompt, representation choice, and the specific notion of “truth” under study.