Papers
Topics
Authors
Recent
Search
2000 character limit reached

Linear Truth Probes Overview

Updated 5 July 2026
  • Linear truth probes are linear classifiers defined on LLM hidden activations that identify a truth direction separating true from false statements.
  • They employ methods like difference-in-mean, logistic regression, and mass-mean to derive affine scoring functions applied across various tasks and contexts.
  • Empirical studies show robust truth separability under controlled settings but note limitations from layer dependence, context sensitivity, and prompt variations.

Searching arXiv for papers on linear truth probes and related truth-direction work. Linear truth probes are linear classifiers defined on hidden activations of LLMs that seek a direction in activation space separating representations associated with true statements from those associated with false statements. In the formulation emphasized by Marks and Tegmark, a probe is a single direction in the model’s activation space that approximately separates true from false statements, with the basic score given by a projection of a hidden-state vector onto that direction (Marks et al., 2023). Subsequent work has generalized the same paradigm to affine scoring functions of the form s(x)=wh(x)+bs(x)=w^\top h(x)+b, trained by logistic regression, linear SVM, mean-difference, or mass-mean procedures, and has studied their consistency across model scales, logical transformations, question answering, conversational formats, and task domains (Bao et al., 1 Jun 2025). The resulting literature presents both positive and negative conclusions: in some sufficiently capable models and narrowly controlled settings, truthfulness behaves like a salient geometric feature; in other settings, the “geometry of truth” is layer-dependent, prompt-sensitive, context-sensitive, or task-specific rather than universal (Azizian et al., 10 Jun 2025).

1. Definition and formalism

A linear truth probe operates on a hidden representation h(x)Rdh(x)\in\mathbb{R}^d extracted from a chosen layer and token position. The core hypothesis is that, in a sufficiently capable LLM, truthfulness is encoded as a roughly one-dimensional feature in the model’s high-dimensional activation space, so that there exists a truth direction wRdw\in\mathbb{R}^d and bias bRb\in\mathbb{R} with score

s(x)=wh(x)+b.s(x)=w^\top h(x)+b.

Passing s(x)s(x) through a sigmoid yields a probabilistic prediction, and thresholding yields a binary prediction (Bao et al., 1 Jun 2025).

Several probe constructions recur across the literature. In Marks and Tegmark’s formulation, for a labeled dataset D\mathcal{D} of statements with labels y{+1,1}y\in\{+1,-1\} and hidden vectors h(x)h(x), the difference-in-mean direction is

μ+=mean{h(x)y(x)=+1},μ=mean{h(x)y(x)=1},\mu_+ = \mathrm{mean}\{h(x)\mid y(x)=+1\},\qquad \mu_- = \mathrm{mean}\{h(x)\mid y(x)=-1\},

h(x)Rdh(x)\in\mathbb{R}^d0

with score h(x)Rdh(x)\in\mathbb{R}^d1 and prediction h(x)Rdh(x)\in\mathbb{R}^d2 (Marks et al., 2023). The same work also introduces mass-mean probing,

h(x)Rdh(x)\in\mathbb{R}^d3

where h(x)Rdh(x)\in\mathbb{R}^d4 is the covariance of the pooled, centered activations, followed by h(x)Rdh(x)\in\mathbb{R}^d5 (Marks et al., 2023).

Other studies use standard logistic regression or linear SVM. The logistic-regression form is

h(x)Rdh(x)\in\mathbb{R}^d6

optimized by binary cross-entropy,

h(x)Rdh(x)\in\mathbb{R}^d7

or, equivalently, as an expectation over examples (Bao et al., 1 Jun 2025). Linear SVM probes instead solve the primal margin objective

h(x)Rdh(x)\in\mathbb{R}^d8

(Bao et al., 1 Jun 2025).

The same formal pattern appears in work on conversational lie detection, where the probe is written as h(x)Rdh(x)\in\mathbb{R}^d9 with wRdw\in\mathbb{R}^d0-regularized binary cross-entropy, and in work on strategic deception, where a logistic-regression probe is fit to residual-stream activations at a fixed layer (Ichmoukhamedov et al., 14 May 2025, Goldowsky-Dill et al., 5 Feb 2025). Across these settings, the operational content of a linear truth probe is stable: a hyperplane in activation space is used to classify hidden states as corresponding to true versus false, correct versus incorrect, or honest versus deceptive responses.

2. Early evidence for a linear truth direction

The canonical evidence for linear truth probes was organized by Marks and Tegmark into three lines: visualization, transfer, and causal intervention (Marks et al., 2023).

First, visualization via PCA showed that hidden representations for curated true/false datasets such as world-city membership, Spanish–English translation, and numerical comparisons formed two clusters that were nearly linearly separable in the top two principal components (Marks et al., 2023). The same study reported that once the first wRdw\in\mathbb{R}^d1 principal components were removed via projection, almost no further linear probe could distinguish truth, suggesting that nearly all linearly accessible truth information lay in the top principal components (Marks et al., 2023).

Second, transfer experiments were designed by training a probe on wRdw\in\mathbb{R}^d2 of one dataset, evaluating on the held-out wRdw\in\mathbb{R}^d3 IID test split, and then evaluating on whole other datasets. In the reported LLaMA-13B layer-13 residual-stream setting, a probe trained on numerical comparisons achieved wRdw\in\mathbb{R}^d4 accuracy when tested on Spanish–English translation, and mass-mean probing generalized slightly better than vanilla logistic regression and unsupervised CCS (Marks et al., 2023). Probes trained on true/false datasets also vastly outperformed probes trained on “likely” text, which was taken as evidence that the model was not merely separating plausible from implausible language (Marks et al., 2023).

Third, causal intervention experiments attempted to show that the truth direction was causally implicated in the model’s decision to output TRUE versus FALSE. Patching-based localization identified a small group of layers and tokens above which truth was computed, using the quantity

wRdw\in\mathbb{R}^d5

Vector-addition interventions then modified hidden states by

wRdw\in\mathbb{R}^d6

where wRdw\in\mathbb{R}^d7 was a normalized probe direction applied at localized layer-token positions (Marks et al., 2023). The intervention was summarized with the Normalized Indirect Effect,

wRdw\in\mathbb{R}^d8

where wRdw\in\mathbb{R}^d9 indicates that the intervention makes a false statement look as true as a genuine true statement (Marks et al., 2023). In LLaMA-13B, a mass-mean probe trained on cities+neg_cities yielded bRb\in\mathbb{R}0 for falsebRb\in\mathbb{R}1true and bRb\in\mathbb{R}2 for truebRb\in\mathbb{R}3false, whereas a logistic-regression probe on the same data achieved bRb\in\mathbb{R}4, and probes trained on “likely” text had almost no causal effect with bRb\in\mathbb{R}5 (Marks et al., 2023).

These results were interpreted as evidence that, at sufficient scale, LLMs linearly represent the truth or falsehood of factual statements, and that simple difference-in-mean probes can recover directions that are at least as causally implicated as more complex alternatives (Marks et al., 2023).

3. Probe families, geometry, and causal interpretation

The literature distinguishes several families of truth probes, with different tradeoffs between simplicity, interpretability, and robustness. Difference-in-mean and mass-mean probes are described as zero-cost and as identifying a concrete direction bRb\in\mathbb{R}6, unlike a full logistic-regression fit which “hides” the direction in a margin-maximizing ellipsoid (Marks et al., 2023). Under Gaussian assumptions, mass-mean matches logistic regression’s average direction while retaining the interpretability of an explicit vector (Marks et al., 2023).

Alternative probe recovery procedures include CCS, CCR, logistic regression, linear SVM, and, in later work, orthogonal multi-direction constructions. CCS uses pairs of opposing sentences and an unsupervised consistency objective; CCR reflects paired representations across a learned hyperplane; mass-mean probing uses bRb\in\mathbb{R}7; and supervised logistic regression learns bRb\in\mathbb{R}8 by minimizing cross-entropy (Schouten et al., 2024). The presence of these methods reflects a methodological dispute over whether a truth direction is best viewed as a single separating direction, an affine hyperplane, or one element within a richer family of linearly decodable directions.

Causal interpretation has remained central. In Marks and Tegmark, the vector-addition intervention was presented as evidence that the identified direction participates causally in generation decisions (Marks et al., 2023). A related causal design appears in work on context-sensitive belief directions, where a premise embedding bRb\in\mathbb{R}9 is shifted along a belief direction,

s(x)=wh(x)+b.s(x)=w^\top h(x)+b.0

and the resulting change in a later hypothesis representation is measured through the same direction (Schouten et al., 2024). That work reported maximal mean s(x)=wh(x)+b.s(x)=w^\top h(x)+b.1 around s(x)=wh(x)+b.s(x)=w^\top h(x)+b.2–s(x)=wh(x)+b.s(x)=w^\top h(x)+b.3 in the best layers, and argued that belief directions are among the causal mediators in the inference process that incorporates in-context information (Schouten et al., 2024). The same paper also defined premise-effect and several normalized error scores, showing that probe outputs could respond correctly to entailment or contradiction while also being affected by irrelevant context (Schouten et al., 2024).

A more intervention-oriented extension is “Truth Forest,” which replaces a single truth axis with multiple orthogonal probes per attention head. In that framework, the probe matrix s(x)=wh(x)+b.s(x)=w^\top h(x)+b.4 is trained with cross-entropy plus an orthogonality penalty

s(x)=wh(x)+b.s(x)=w^\top h(x)+b.5

and the resulting head-level directions are aggregated with exponential decay weights into a multi-scale ensemble (Chen et al., 2023). Truth Forest also introduces “Random Peek,” which truncates each answer at a random cutoff and extracts head features at the final retained position, motivated by a generating–discerning gap between where truth is represented for classification and where it is useful for generation (Chen et al., 2023). This work suggests that the single-direction picture may be incomplete in intervention settings even when linear separability remains important.

A plausible implication is that “linear truth probes” now denote not one single algorithm but a family of low-complexity readouts, ranging from one-dimensional difference-in-mean directions to multi-direction orthogonal ensembles, all grounded in the claim that truth-related information is linearly accessible in hidden space.

4. Generalization across logical forms, question answering, and context

Later work expanded the original true/false-statement setting to logical transformations, question answering, contextual tasks, and conversational formats. A central result is that probes trained on declarative atomic statements can generalize beyond those training distributions, but only under some conditions (Bao et al., 1 Jun 2025).

In a study of consistency across logical transformations and QA, probes were trained on affirmative atomic statements in six topics and tested on their syntactic negations. Negation generalization, measured by AUROC s(x)=wh(x)+b.s(x)=w^\top h(x)+b.6, improved strongly with model capability: Llama-2-7B showed AUROC s(x)=wh(x)+b.s(x)=w^\top h(x)+b.7 on no topics, Llama-2-13B s(x)=wh(x)+b.s(x)=w^\top h(x)+b.8–s(x)=wh(x)+b.s(x)=w^\top h(x)+b.9 on s(x)s(x)0 topics, Llama-3.1-8B s(x)s(x)1–s(x)s(x)2 on s(x)s(x)3, and Llama-3.1-70B-Instruct s(x)s(x)4–s(x)s(x)5 on s(x)s(x)6 (Bao et al., 1 Jun 2025). The same study found that on Llama-3.1-8B, conjunction AUROC was approximately s(x)s(x)7–s(x)s(x)8 across topics and disjunction AUROC approximately s(x)s(x)9–D\mathcal{D}0 (Bao et al., 1 Jun 2025). On QA tasks, probes trained on atomic statements generalized to MMLU and TriviaQA with AUROC, ECE, and Brier Score reported across zero-shot and few-shot settings; on contextual tasks, SciQ zero-shot reached AUROC D\mathcal{D}1, BoolQ with options AUROC D\mathcal{D}2, and XSum zero-shot AUROC D\mathcal{D}3, improving to D\mathcal{D}4 under TTT prompting (Bao et al., 1 Jun 2025).

The same paper described a practical selective question-answering procedure: for multiple sampled answers D\mathcal{D}5 to question D\mathcal{D}6, the probe scores each concatenated D\mathcal{D}7 pair as D\mathcal{D}8, converts to D\mathcal{D}9, and retains only those answers with y{+1,1}y\in\{+1,-1\}0 (Bao et al., 1 Jun 2025). On TriviaQA with 20 sampled answers for Llama-3.1-8B and an SVM probe, overall sample accuracy was y{+1,1}y\in\{+1,-1\}1, the probe labeled approximately y{+1,1}y\in\{+1,-1\}2 as true, and accuracy among retained answers rose to y{+1,1}y\in\{+1,-1\}3, a gain of about 9 points (Bao et al., 1 Jun 2025).

Generalization in conversational settings is more fragile. In work on conversational formats, probes trained on short assistant–user dialogues ending in a true or false statement generalized well to other short formats, but poorly to longer formats in which the lie appeared earlier in the input prompt (Ichmoukhamedov et al., 14 May 2025). For Llama-3-8b-Instruct at layer 18, y{+1,1}y\in\{+1,-1\}4 accuracy was y{+1,1}y\in\{+1,-1\}5, whereas y{+1,1}y\in\{+1,-1\}6 dropped to y{+1,1}y\in\{+1,-1\}7 (Ichmoukhamedov et al., 14 May 2025). Appending the fixed key phrase “Please determine whether any statement above is false. Answer Yes/No.” substantially improved this cross-format transfer, with average y{+1,1}y\in\{+1,-1\}8 accuracy at layer 18 rising from y{+1,1}y\in\{+1,-1\}9 to h(x)h(x)0 for Llama and from h(x)h(x)1 to h(x)h(x)2 for Mistral (Ichmoukhamedov et al., 14 May 2025). PCA visualizations were used to show that the key phrase preserved the geometry of true/false separability as conversation length increased (Ichmoukhamedov et al., 14 May 2025).

Context sensitivity adds a further complication. Probes can show non-trivial responsiveness to supporting or contradicting premises, but also large sensitivity to irrelevant or corrupted context, and the error pattern depends on layer, model type, and data (Schouten et al., 2024). This suggests that truth directions may capture conditional beliefs in context while also being vulnerable to spurious prompt effects.

5. Limits of universality: task dependence, layers, prompts, and orthogonality

A major controversy in the area concerns whether there exists a universal truth direction. Several studies argue that claims of universality are substantially limited (Azizian et al., 10 Jun 2025, Poulis et al., 4 Apr 2026).

One line of criticism examines cross-task transfer for probes trained to distinguish correct versus incorrect answers on generated outputs. In work on “The Geometries of Truth Are Orthogonal Across Tasks,” probes were trained separately on TriviaQA, NaturalQuestions, SimpleQA, SQuAD, BioASQ, GSM8K, and SVAMP, using hidden activations at a fixed layer and token position (Azizian et al., 10 Jun 2025). The average cosine similarity between task-specific probe weights was often below h(x)h(x)3, with only closely related task pairs such as TriviaQA and NQ reaching cosine similarity around h(x)h(x)4 (Azizian et al., 10 Jun 2025). The Pearson correlation between cross-task AUROC boost and cosine similarity was h(x)h(x)5 with h(x)h(x)6 (Azizian et al., 10 Jun 2025). When h(x)h(x)7-regularized probes were used, support overlap was below h(x)h(x)8 for most task pairs, and even the best pair, TriviaQA versus NQ, overlapped only about h(x)h(x)9 (Azizian et al., 10 Jun 2025). Training on unions of tasks or with a mixture-of-experts did not recover a universal direction: for Qwen-2.5 on TriviaQA, in-domain AUROC was μ+=mean{h(x)y(x)=+1},μ=mean{h(x)y(x)=1},\mu_+ = \mathrm{mean}\{h(x)\mid y(x)=+1\},\qquad \mu_- = \mathrm{mean}\{h(x)\mid y(x)=-1\},0, NQμ+=mean{h(x)y(x)=+1},μ=mean{h(x)y(x)=1},\mu_+ = \mathrm{mean}\{h(x)\mid y(x)=+1\},\qquad \mu_- = \mathrm{mean}\{h(x)\mid y(x)=-1\},1TriviaQA was μ+=mean{h(x)y(x)=+1},μ=mean{h(x)y(x)=1},\mu_+ = \mathrm{mean}\{h(x)\mid y(x)=+1\},\qquad \mu_- = \mathrm{mean}\{h(x)\mid y(x)=-1\},2, training on all other six tasks yielded μ+=mean{h(x)y(x)=+1},μ=mean{h(x)y(x)=1},\mu_+ = \mathrm{mean}\{h(x)\mid y(x)=+1\},\qquad \mu_- = \mathrm{mean}\{h(x)\mid y(x)=-1\},3, and mixture-of-experts yielded μ+=mean{h(x)y(x)=+1},μ=mean{h(x)y(x)=1},\mu_+ = \mathrm{mean}\{h(x)\mid y(x)=+1\},\qquad \mu_- = \mathrm{mean}\{h(x)\mid y(x)=-1\},4 (Azizian et al., 10 Jun 2025).

Another line of criticism focuses on layer, task-type, and prompt dependence. “Testing the Limits of Truth Directions in LLMs” showed that truth directions are highly layer-dependent and differ between factual retrieval and reasoning tasks (Poulis et al., 4 Apr 2026). On Llama-3.1-8B-Instruct, factual tasks F0–F3 reached AUROC μ+=mean{h(x)y(x)=+1},μ=mean{h(x)y(x)=1},\mu_+ = \mathrm{mean}\{h(x)\mid y(x)=+1\},\qquad \mu_- = \mathrm{mean}\{h(x)\mid y(x)=-1\},5 by layers 8–12, whereas counting tasks F4 and F5 peaked much later, around layers 18–22, and often below perfect; arithmetic A1 neared 1.0 around layer 22, A2 around layer 25, and A3 only reached about μ+=mean{h(x)y(x)=+1},μ=mean{h(x)y(x)=1},\mu_+ = \mathrm{mean}\{h(x)\mid y(x)=+1\},\qquad \mu_- = \mathrm{mean}\{h(x)\mid y(x)=-1\},6 by layer 30+ (Poulis et al., 4 Apr 2026). Early-layer probes trained on F0 could invert true and false on F1 negation with AUROC μ+=mean{h(x)y(x)=+1},μ=mean{h(x)y(x)=1},\mu_+ = \mathrm{mean}\{h(x)\mid y(x)=+1\},\qquad \mu_- = \mathrm{mean}\{h(x)\mid y(x)=-1\},7, only recovering by mid-layers, which the authors interpreted as evidence that early layers capture polarity cues rather than truth (Poulis et al., 4 Apr 2026). Prompting also changed the geometry: cosine similarity between no-prompt and ask-correct truth directions was low, approximately μ+=mean{h(x)y(x)=+1},μ=mean{h(x)y(x)=1},\mu_+ = \mathrm{mean}\{h(x)\mid y(x)=+1\},\qquad \mu_- = \mathrm{mean}\{h(x)\mid y(x)=-1\},8–μ+=mean{h(x)y(x)=+1},μ=mean{h(x)y(x)=1},\mu_+ = \mathrm{mean}\{h(x)\mid y(x)=+1\},\qquad \mu_- = \mathrm{mean}\{h(x)\mid y(x)=-1\},9, and a no-prompt probe tested on ask-correct activations suffered AUROC drops of about h(x)Rdh(x)\in\mathbb{R}^d00–h(x)Rdh(x)\in\mathbb{R}^d01 on arithmetic tasks at their emergence layers (Poulis et al., 4 Apr 2026).

A related critique concerns model capability. The 2025 generalization study concluded that not all LLMs exhibit consistent truth directions, with stronger representations observed in more capable models, particularly in logical negation (Bao et al., 1 Jun 2025). This point aligns with Marks and Tegmark’s conclusion that the evidence is strongest “at sufficient scale” (Marks et al., 2023).

Taken together, these findings oppose a strong universality thesis. Linear truth probes can be highly effective within controlled domains or in sufficiently capable models, but the geometry they exploit can be task-dependent, layer-dependent, and prompt-sensitive rather than invariant across settings.

6. Beyond binary truth: deception, “neither,” and robustness under shift

The truth-probe paradigm has also been extended from binary factual correctness to deception monitoring and ternary veracity classification. These extensions preserve the use of linear readouts but alter the target phenomenon.

In work on strategic deception, a logistic-regression probe was trained on token-level activations from layer 22 of Llama-3.3-70B-Instruct using two synthetic training sets: a contrasting-instructions dataset of 10,000 honest/deceptive pairs and a roleplaying scenarios dataset graded on a 1–7 honesty scale (Goldowsky-Dill et al., 5 Feb 2025). On evaluation datasets, the instructed-pairs-trained probe achieved AUROC h(x)Rdh(x)\in\mathbb{R}^d02 on Roleplaying, h(x)Rdh(x)\in\mathbb{R}^d03 on Insider Trading Report, h(x)Rdh(x)\in\mathbb{R}^d04 on Insider Trading Confirmation, and h(x)Rdh(x)\in\mathbb{R}^d05 on Sandbagging, with recall at h(x)Rdh(x)\in\mathbb{R}^d06 false positive rate ranging from h(x)Rdh(x)\in\mathbb{R}^d07 to h(x)Rdh(x)\in\mathbb{R}^d08 (Goldowsky-Dill et al., 5 Feb 2025). The study emphasized that output monitoring alone is insufficient because deceptive models may produce seemingly benign text while internal reasoning remains misaligned (Goldowsky-Dill et al., 5 Feb 2025).

Pressure-testing work complicates this optimistic picture. A 2026 study across Gemma 3 models examined four hypotheses about deception encoding: a single linear direction, a low-dimensional symmetric subspace, a convex conic hull, and an entropy proxy (Kumar, 27 May 2026). The single-direction hypothesis was rejected, with one-dimensional probes achieving only AUROC h(x)Rdh(x)\in\mathbb{R}^d09–h(x)Rdh(x)\in\mathbb{R}^d10 while full probes exceeded h(x)Rdh(x)\in\mathbb{R}^d11 on clean data (Kumar, 27 May 2026). Deception did not form a statistically significant low-dimensional linear subspace under the paper’s permutation-null test, yet multi-dimensional probes with h(x)Rdh(x)\in\mathbb{R}^d12 recovered AUROC at or above h(x)Rdh(x)\in\mathbb{R}^d13, which the authors interpreted as a distributed, sub-threshold encoding (Kumar, 27 May 2026). Vanilla probes collapsed under stylistic shifts, with mean AUROC on held-out styles falling to h(x)Rdh(x)\in\mathbb{R}^d14 for Gemma 3 27B, while style-augmented training restored held-out mean AUROC to h(x)Rdh(x)\in\mathbb{R}^d15 at 27B and h(x)Rdh(x)\in\mathbb{R}^d16 at 4B (Kumar, 27 May 2026). This was presented as evidence that apparent fragility can be a distributional-narrowness artifact rather than a fundamental architectural limitation (Kumar, 27 May 2026).

A different extension is ternary veracity. “The Trilemma of Truth in LLMs” argues that LLMs capture a third type of signal distinct from true and false and neither true nor false (Savcisens et al., 30 Jun 2025). The paper introduces sAwMIL, a sparse-aware multiple-instance learning method with conformal prediction, evaluating five validity criteria—correlation, generalization, selectivity, manipulation, and locality—across 16 open-source LLMs and three datasets (Savcisens et al., 30 Jun 2025). Its multiclass version trains one-vs-all probes for true, false, and neither, combines them via softmax, and calibrates outputs with split-conformal prediction (Savcisens et al., 30 Jun 2025). Across 16 models, multiclass sAwMIL achieved average W-MCC approximately h(x)Rdh(x)\in\mathbb{R}^d17 on City Locations, h(x)Rdh(x)\in\mathbb{R}^d18 on Medical Indications, and h(x)Rdh(x)\in\mathbb{R}^d19 on Word Definitions, with chat models outperforming default models by about h(x)Rdh(x)\in\mathbb{R}^d20–h(x)Rdh(x)\in\mathbb{R}^d21 W-MCC (Savcisens et al., 30 Jun 2025). The same study reported that the veracity signal is often concentrated in the third quarter of an LLM’s depth and that truth and falsehood signals are not always symmetric (Savcisens et al., 30 Jun 2025).

These developments broaden the scope of linear truth probes from simple true/false factuality to internal belief monitoring, deception detection, abstention, and uncertainty-aware classification. They also indicate that binary linear separation is not always the right abstraction.

7. Interpretive significance, mechanisms, and open problems

Several papers connect linear truth probes to broader questions about internal world models and the emergence of linear representations. Marks and Tegmark interpret the existence of a stable truth direction as suggesting that LLMs build a world-model that encodes factual correctness as a mostly linear feature (Marks et al., 2023). A later mechanistic study introduces a one-layer transformer toy model in which truth encoding emerges because factual statements co-occur with other factual statements, so representing a latent truth bit lowers language-modeling loss on future tokens (Ravfogel et al., 17 Oct 2025). In that toy setting, learning exhibited two phases: rapid memorization of factual associations followed by a slower phase in which true versus false contexts became linearly separable after layer normalization, reducing LM loss (Ravfogel et al., 17 Oct 2025). The same work reported AUC rising from about h(x)Rdh(x)\in\mathbb{R}^d22 to about h(x)Rdh(x)\in\mathbb{R}^d23 around batch 7,500 in the toy model, and AUC above h(x)Rdh(x)\in\mathbb{R}^d24 from middle layers onward when probing pretrained Llama-3-8B and Pythia 6.9B under CounterFact-based settings (Ravfogel et al., 17 Oct 2025).

Other work suggests that a bag of linear directions may itself be a projection of more structured latent organization. “Tensor Product Representation Probes Reveal Shared Structure Across Linear Directions,” though developed in the Othello setting rather than factual truth evaluation, argues that directional representations may be projections of more structured underlying representations (Lee et al., 11 May 2026). This suggests that linear truth directions could reflect a compressed readout of richer relational representations, although the paper does not study factual truth directly.

The main limitations are consistently framed. Results are often restricted to simple, unambiguous factual statements rather than nuance, controversial claims, or multi-hop inference (Marks et al., 2023). Bias terms and thresholds can remain under-determined and require calibration (Marks et al., 2023). Prompt context, conversational format, and irrelevant supporting text can alter probe outputs substantially (Schouten et al., 2024, Ichmoukhamedov et al., 14 May 2025). Harder reasoning tasks such as counting or multi-operator arithmetic are less linearly separable than simple retrieval tasks (Poulis et al., 4 Apr 2026). Task-specific geometries can be nearly orthogonal, limiting transfer across domains (Azizian et al., 10 Jun 2025). For some RLHF- or distillation-tuned models, nonlinear probes may be required to capture veracity signals (Savcisens et al., 30 Jun 2025).

The overall research picture is therefore mixed but coherent. Linear truth probes are a well-defined and empirically productive methodology for reading out truth-related information from LLM activations. In sufficiently capable models and controlled domains, a simple direction or hyperplane can support strong generalization and can even be causally implicated in model outputs (Marks et al., 2023, Bao et al., 1 Jun 2025). At the same time, universality claims are constrained by layer dependence, prompt sensitivity, context effects, task orthogonality, and distribution shift (Azizian et al., 10 Jun 2025, Poulis et al., 4 Apr 2026). A plausible synthesis is that linearly accessible veracity features often exist, but their stability depends on model capability, dataset design, layer selection, and the semantic structure of the task rather than on a single universal “truth direction” that transfers unchanged across all settings.

Topic to Video (Beta)

No one has generated a video about this topic yet.

Whiteboard

No one has generated a whiteboard explanation for this topic yet.

Follow Topic

Get notified by email when new papers are published related to Linear Truth Probes.