Superficial Belief in AI: Shallow Model Insights

• Superficial belief is a concept describing how models depend on easily perturbed, surface-level regularities rather than deep, semantically integrated knowledge.
• Empirical studies show that minor textual transformations cause significant degradation in model truthfulness and decision accuracy, highlighting the brittleness of such representations.
• The phenomenon spans LLM interpretability, decision-making, alignment, and human–AI interaction, underlining the need for robust, causally grounded model architectures.

Superficial belief is a family of concepts used across machine learning, large-language-model interpretability, alignment, decision theory, and human–AI interaction to denote belief-like structure that is organized around shallow regularities rather than robust, semantically integrated, or causally grounded understanding. In contemporary LLM work, the term has been used for at least three closely related phenomena: truthfulness representations that depend on lexical, orthographic, or formatting resemblance to pre-training data and collapse under innocuous perturbations (Haller et al., 13 Oct 2025); decision behavior that is systematic enough to support a recovered latent priority structure, while explicit reasons only partially track that structure (Freedman et al., 9 Jun 2026); and alignment behavior that can be reproduced by shallow changes to the final token-selection layer without modifying deeper transformer representations (Chen et al., 7 Feb 2025). Related antecedents study the removal of superficial statistics from learned representations (Wang et al., 2019), the rational formation of superficial posteriors under costly information processing (Vaccari, 2024), and the human tendency to place heuristic trust in AI predictions under “rational superstition” (Lee et al., 2024).

1. Conceptual scope and competing definitions

The term is not used uniformly. In "LLM Knowledge is Brittle: Truthfulness Representations Rely on Superficial Resemblance," Haller et al. define a “superficial belief” as an LLM’s internal encoding of facts or truthfulness that is overly tied to the exact lexical, orthographic, or formatting patterns seen during pre-training, such that innocuous changes to those surface features cause the model’s internal judgment of truth to collapse (Haller et al., 13 Oct 2025). Their contrast class is a stable, meaning-centered concept of truth.

In "Superficial Beliefs in LLM Decision-Making," Freedman and Toni use a different but related notion. Drawing on a superficialist view in philosophy, they treat belief as something attributable from stable, structured outward behavior rather than from assumptions about internal architecture. In their operationalization, an LLM exhibits a weak, decision-local form of belief if its choices across similar problems are systematic enough to support a probabilistic revealed preference over visible attributes, while its verbalized reasons only partially track those revealed preferences (Freedman et al., 9 Jun 2026).

In "Extracting and Understanding the Superficial Knowledge in Alignment," Chen et al. define superficial knowledge as “knowledge that can be acquired through easily token restyling, without affecting the model’s ability to capture underlying causal relationships between tokens.” Here the defining feature is architectural shallowness: the aligned behavior can be approximated by modifying the final projection matrix while leaving the transformer backbone fixed (Chen et al., 7 Feb 2025).

Outside LLM interpretability, Vaccari studies “superficial” beliefs as posteriors formed after observing only a costless first-stage signal rather than paying to process a second informative component; these beliefs are shallow not because they are irrational, but because full processing is costly (Vaccari, 2024). In human–AI interaction, de Oliveira Sant’Anna et al. describe a related phenomenon as “rational superstition”: belief in AI predictions driven more by mental heuristics and intuition than by critical evaluation (Lee et al., 2024). Across these literatures, the common denominator is not mere error. It is the dependence of belief-like outputs on surface regularities, local heuristics, or low-cost processing in place of robust abstraction.

2. Superficial belief as brittle truthfulness representation in LLMs

Haller et al. investigate whether the latent encoding of truthfulness in decoder-only LLMs is robust under semantically preserving out-of-distribution transformations. They define the hidden-state extractor as $h = f_\theta(x) \in \mathbb{R}^n$, where $x$ is a tokenized statement and $h$ is the final-token residual-stream activation. Truthfulness is said to be internally represented if there exists a linear or non-linear decision boundary in $\mathbb{R}^n$ that reliably assigns $f_\theta(x)$ to “true” or “false” (Haller et al., 13 Oct 2025).

Their methodology perturbs true and false statements by typos and punctuation noise via AugLy, syntactic negation via the negate library, Yoda-speak clause reordering via NL-Augmenter, and translation into French or Spanish via NLLB-200. They evaluate three probes: a linear classifier $w^\top h + b$, a 3-layer non-linear MLP with hidden units 256–128–64 and ReLU activations, and $P(\mathrm{True})$, an output-based method that normalizes next-token probabilities for “(A) correct” versus “(B) incorrect” in a 6-shot multiple-choice prompt. All probes are scored by area under the receiver-operating curve, and the signed linear margin is $w^\top f_\theta(x) + b$.

The experimental setup covers OLMo, OLMo-2, Llama 3.1 Instruct, Llama 3.2 Instruct, and Gemma 3 models; the detailed setup lists four benchmarks: True-False statements, MMLU, OpenBookQA, and TruthfulQA. Activations are extracted from six candidate layers for larger models, with best-layer selection on the untransformed split. The MLP and logistic probe are trained for 5 epochs with Adam at learning rate $10^{-2}$, using stratified 6-fold cross-validation and balanced true/false sets (Haller et al., 13 Oct 2025).

The central empirical result is that, across every model, dataset, and probe, truth-separability degrades sharply as transformations increase statement perplexity and thus “OOD-ness.” On True-False for Llama 3.1 8B with the non-linear probe, untransformed data yield AUC $\approx 0.98$, but under typos and punctuation noise AUC falls linearly with average perplexity, with standardized regression slope $x$0; $x$1 degrades faster at $x$2. On MMLU with the non-linear probe, untransformed AUC is $x$3 and the slope is $x$4, the steepest among the four datasets. On True-False, most models degrade at $x$5 in $x$6, with Llama 3.1 70B the worst case at $x$7 and Gemma 3 4B the mildest at $x$8 (Haller et al., 13 Oct 2025).

Transformation-specific effects sharpen the diagnosis. For typos, punctuation, and Yoda transformations, $x$9. Translation causes dramatic AUC damage even when $h$0, making perplexity a false negative as an OOD proxy. Negation produces no $h$1 and no $h$2 for latent probes, indicating surprising invariance of the internal representation to truth-value flipping, but $h$3 still degrades, pointing to output-side brittleness not mirrored in the latent space. The paper further reports nearly parallel degradation slopes on the correctly answered MMLU subset and on the full set, implying that even benchmark-correct outputs need not rest on robust truthfulness representations (Haller et al., 13 Oct 2025).

Within this usage, superficial belief is therefore an internal truth signal that is high-performing in-distribution yet highly dependent on exact surface form. A plausible implication is that strong probe performance on unperturbed benchmarks is insufficient evidence for genuinely abstract factual knowledge.

3. Superficial belief in LLM decision-making

Freedman and Toni study a different problem: whether LLMs merely imitate reasons when choosing between two options, or whether their choices reveal a systematic underlying decision structure. Their synthetic binary decision tasks present two profiles, $h$4 and $h$5, each defined by four graded attributes $h$6 with levels in $h$7 encoded as $h$8. Profiles are sampled so that neither dominates the other. The dataset contains 400 training source problems and 100 held-out source problems, and each source problem is rendered under four prompt variants combining two attribute-ordering permutations and two label flips, yielding 1,600 training prompts and 400 test prompts per theme. Themes are Drugs, Policy, Software, and two control variants of Drugs where one attribute is irrelevant. Each rendered prompt is sampled three times under a fixed temperature/top-$h$9 setting (Freedman et al., 9 Jun 2026).

For each theme and model setting, they fit a binomial logistic regression on the training split. If $\mathbb{R}^n$0 is the number of times out of 3 the model chose A on rendered prompt $\mathbb{R}^n$1, and $\mathbb{R}^n$2 is the attribute difference, then

$\mathbb{R}^n$3

On test prompts, the recovered per-attribute contributions are $\mathbb{R}^n$4, and the revealed driver $\mathbb{R}^n$5 is the attribute with maximal contribution if the observed choice is A, or minimal contribution if the observed choice is B (Freedman et al., 9 Jun 2026).

The behavioral surrogate predicts held-out choices well: aggregated over all themes and model settings, held-out choice prediction accuracy is $\mathbb{R}^n$6 with Wilson 95% confidence interval $\mathbb{R}^n$7, and held-out negative log-likelihood is $\mathbb{R}^n$8 bits per prompt. Simple heuristics are materially weaker: the equal-weight additive rule reaches $\mathbb{R}^n$9 and the count-better rule $f_\theta(x)$0. This indicates that model behavior is systematically related to visible attribute differences rather than being random (Freedman et al., 9 Jun 2026).

The crucial finding is a gap between behavioral structure and introspective access. In direct responses, choice alignment with the behavioral prediction is $f_\theta(x)$1, but attribute alignment between the stated “most important attribute” and the revealed driver is only $f_\theta(x)$2 with confidence interval $f_\theta(x)$3. In a separate score-based judge prompt, reconstructed choices align with the behavioral choice at $f_\theta(x)$4, while recovered driver alignment is $f_\theta(x)$5 with confidence interval $f_\theta(x)$6. Theme-level rates vary modestly, and in control themes the irrelevant attribute is chosen in direct reports less than $f_\theta(x)$7 of the time and appears in recovered drivers less than $f_\theta(x)$8 of the time, which rules out arbitrary responding (Freedman et al., 9 Jun 2026).

A dense robustness program leaves the qualitative picture intact. Within each of the 100 held-out source problems there are 12 realizations from prompt-order and sampling variation. Pairwise reproducibility is $f_\theta(x)$9 for direct choices and $w^\top h + b$0 for direct attributes; for the judge it is $w^\top h + b$1 for choices and $w^\top h + b$2 for attributes. “Majority” alignment is $w^\top h + b$3 for direct choices but only $w^\top h + b$4 for direct attributes, and $w^\top h + b$5 and $w^\top h + b$6 respectively for the judge. Alternative behavioral models change held-out choice accuracy by less than $w^\top h + b$7, while driver alignment remains around $w^\top h + b$8. In a structurally varied 6-attribute hospital cyber-response task, behavioral choice accuracy reaches $w^\top h + b$9 for GPT-5-mini NT and $P(\mathrm{True})$0 for Qwen3 NT, yet direct attribute alignment is only $P(\mathrm{True})$1 and $P(\mathrm{True})$2, and judge attribute alignment $P(\mathrm{True})$3 and $P(\mathrm{True})$4 (Freedman et al., 9 Jun 2026).

Under this interpretation, superficial belief is neither arbitrary behavior nor fully articulated belief. The model behaves as if guided by probabilistic local priorities over attributes, but has only limited verbal access to the attributes that drive its decisions.

4. Superficial knowledge in alignment and the contrast with belief depth

Chen et al. analyze superficial knowledge in aligned LLMs through an explicitly architectural lens. Let $P(\mathrm{True})$5 denote the transformer backbone and final projection of the aligned model, and $P(\mathrm{True})$6 those of the base model. At generation step $P(\mathrm{True})$7, the hidden state is $P(\mathrm{True})$8 and logits are $P(\mathrm{True})$9. Superficial knowledge is the component of alignment that can be reproduced by a shallow residual shift $w^\top f_\theta(x) + b$0 added to the base model’s final projection while keeping the transformer backbone fixed, so that

$w^\top f_\theta(x) + b$1

closely matches the aligned-model logits $w^\top f_\theta(x) + b$2 (Chen et al., 7 Feb 2025).

Their extraction method freezes $w^\top f_\theta(x) + b$3 and $w^\top f_\theta(x) + b$4, initializes $w^\top f_\theta(x) + b$5 randomly, and minimizes the KL divergence between the aligned-model token distribution and the base-plus-residual token distribution over alignment data:

$w^\top f_\theta(x) + b$6

The resulting model is the “base+superficial” model. Across GSM, Toxigen, Advbench, and TruthfulQA, the paper quantifies the “superficial portion” of alignment as the fraction of aligned-model improvement over base that is recovered by this shallow residual (Chen et al., 7 Feb 2025).

On the LLaMA2-7B results highlighted in the paper, superficial knowledge accounts for a large share of alignment behavior. GSM accuracy rises from $w^\top f_\theta(x) + b$7 to $w^\top f_\theta(x) + b$8 under full alignment, and the superficial residual recovers $w^\top f_\theta(x) + b$9, approximately $10^{-2}$0 of the aligned gain. On Toxigen, toxicity falls from $10^{-2}$1 to $10^{-2}$2, and the superficial model also reaches $10^{-2}$3, corresponding to $10^{-2}$4 coverage. On Advbench, HarmRate falls from $10^{-2}$5 to $10^{-2}$6, again with $10^{-2}$7 recovery by the superficial model. On TruthfulQA, the score rises from $10^{-2}$8 to $10^{-2}$9, and the superficial model reaches $\approx 0.98$0, approximately $\approx 0.98$1 of the factual gain (Chen et al., 7 Feb 2025).

At token level, the distinction between shallow and deep components appears position-dependent. The paper reports that token positions 1–10 are almost fully explained by superficial head shifts, with KL divergence to the aligned model approaching zero, whereas later tokens retain residual divergence. The worked example is a math word problem in which the base+superficial model adopts the aligned model’s step-by-step style but still miscalculates “204 + 160 + 330 = 894,” whereas the aligned model outputs 694. This is presented as evidence that token restyling and safe response formatting can be housed in the output head, while arithmetic integration and related causal-relational knowledge require deeper transformer modifications (Chen et al., 7 Feb 2025).

Chen et al. also show two practical consequences. First, a “black-box” superficial transform distilled in logits space from LLaMA2-7B-Chat transfers to LLaMA2-13B, improving GSM accuracy from $\approx 0.98$2 to $\approx 0.98$3, reducing HarmRate to $\approx 0.98$4, and raising TruthfulQA from $\approx 0.98$5 to $\approx 0.98$6. Second, after a fine-tuning attack that raises HarmRate from $\approx 0.98$7 to $\approx 0.98$8, plugging in the previously extracted superficial head reduces HarmRate to $\approx 0.98$9, restoring $x$00 of the safety gain while leaving MMLU accuracy unchanged (Chen et al., 7 Feb 2025).

A complementary perspective comes from "Believe It or Not: How Deeply do LLMs Believe Implanted Facts?" which operationalizes belief depth along generalization, robustness, and representational similarity. Prompting and mechanistic editing via AlphaEdit produce shallow, brittle beliefs: prompting can succeed on direct questioning but collapses under scrutiny, and AlphaEdit fails almost entirely on generalization, robustness, and representational similarity. By contrast, Synthetic Document Finetuning (SDF) on Llama 3.3 70B Instruct attains 60–80 percent alignment on downstream tasks, causal implications, and Fermi estimates for plausible facts; retains high belief rates under adversarial prompts, critique tasks, debate, and inference-time scaling up to 1,200 reasoning tokens; and in standard truth probing causes 60–70% probe error, with highly plausible AKC and BKC SDF-facts remaining linearly indiscriminable from genuine knowledge in adversarial probing (Slocum et al., 20 Oct 2025). This contrast is important: it distinguishes superficial or shallow behavioral changes from deeper edits that behave more like pre-trained knowledge, while also showing that even SDF remains brittle for egregious falsehoods such as inverse-cube gravity.

5. Antecedents beyond LLM belief attribution

Before the recent LLM literature, Wang et al. framed superficiality in representation-learning terms. "Learning Robust Representations by Projecting Superficial Statistics Out" identifies texture-sensitive gray-level co-occurrence matrix features as a family of superficial signals that can be extracted by a differentiable NGLCM module and then suppressed either adversarially, using a reverse-gradient texture predictor, or by orthogonal projection through HEX. In the HEX formulation, the joint logits $x$01 are projected orthogonally to the column space of the texture-only logits $x$02:

$x$03

Across several domain-generalization benchmarks, these methods improve robustness under distribution shift. On MNIST-Rotation, HEX achieves an average of $x$04 versus $x$05 for CrossGrad and $x$06 for ADV; on PACS, HEX reaches $x$07 on Art and $x$08 on Cartoon, while the average is $x$09, close to Fusion at $x$10 (Wang et al., 2019). In this earlier usage, the target is not belief per se, but the broader problem of disentangling semantic from superficial statistics.

Vaccari extends the concept to Bayesian decision-making with costly information acquisition. The decision-maker first observes a costless signal component $x$11, may pay cost $x$12 to observe $x$13, and then acts. The superficial posterior after observing only $x$14 is

$x$15

and full processing is chosen iff $x$16. The model shows that polarization, apparent confirmation bias, apparent disconfirmation bias, under-reaction, and over-reaction can arise under standard Bayesian updating once processing costs are included. In the paper’s numerical illustration with $x$17, $x$18, $x$19, $x$20, and $x$21, the agent stops after $x$22 because $x$23, yielding $x$24, but pays after $x$25 because $x$26; if $x$27, the posterior becomes $x$28 (Vaccari, 2024). Superficial belief here is not a representational defect; it is an optimal stopping point under cost constraints.

In human–AI interaction, de Oliveira Sant’Anna et al. analyze belief in AI predictions as a kind of “rational superstition.” In an experiment with 238 participants, fictitious predictions were attributed to AI, astrology, or personality psychology, and participants rated validity, reliability, usefulness, and personalization. A multiple regression predicting overall AI believability from astrology and personality believability plus controls achieved $x$29, adjusted $x$30, with $x$31 for astrology believability and $x$32 for personality believability, both with $x$33. Paranormal belief increased perceived validity, reliability, and usefulness of AI predictions; positive AI attitudes increased perceived validity by $x$34 and reliability by $x$35; conscientiousness decreased perceived validity by $x$36 across sources; and interest in the topic increased perceived validity by $x$37 per point. The study reports no evidence that cognitive style affects belief in fictitious AI-generated predictions in the expected skeptical direction (Lee et al., 2024). This is an adjacent but important extension: superficial belief can characterize not only model internals, but also human trust in model outputs.

6. Interpretive significance and open problems

Several common claims are challenged by this literature. High in-distribution separability of true versus false statements does not by itself establish robust, meaning-centered knowledge, because separability can collapse under semantically preserving surface perturbations (Haller et al., 13 Oct 2025). Coherent verbal rationales do not by themselves identify what drove a model’s choice, because direct reports and judge-style reconstructions recover the behaviorally inferred driver only partially (Freedman et al., 9 Jun 2026). Strong safety or style improvements after alignment do not by themselves imply deep internal change, because a substantial share of those gains may reside in shallow modifications to the final projection head (Chen et al., 7 Feb 2025).

The resulting methodological lesson is caution toward introspective and probe-based evidence. Haller et al. explicitly argue that truthfulness probes should be treated with caution because high in-distribution separability does not guarantee generalizable knowledge (Haller et al., 13 Oct 2025). Freedman and Toni similarly conclude that reproducibility of verbal reports does not guarantee correspondence to the attribute that behaviorally drove the decision, and suggest that richer introspective architectures or alternative measurement criteria such as coherence, use, and uniformity may be needed (Freedman et al., 9 Jun 2026). Chen et al. imply that alignment evaluation must distinguish shallow token restyling from deeper causal-relational competence, especially in reasoning-heavy settings (Chen et al., 7 Feb 2025). The belief-depth framework sharpens this further by proposing generalization, robustness, and representational similarity as joint criteria for whether an edited fact behaves like genuine knowledge (Slocum et al., 20 Oct 2025).

The open problems identified across these works are closely aligned. Haller et al. propose both data-centric and method-centric directions: increasing variability and paraphrase richness in pre-training and fine-tuning corpora, curating more diverse linguistic forms such as non-standard word orders and orthographic noise, developing training objectives that explicitly penalize surface sensitivity, exploring latent-space paraphrase contrastive learning, and moving beyond perplexity to alternative OOD proxies such as log-average $x$38-gram counts or semantic embeddings (Haller et al., 13 Oct 2025). They also pose the unresolved question of how to disentangle brittleness caused by pre-training data scarcity from brittleness caused by architectural inductive biases. Chen et al. add that the non-superficial component of alignment is multi-faceted and remains hard to isolate (Chen et al., 7 Feb 2025). The SDF results suggest that deeper belief implantation is possible, but not universal: facts that strongly contradict basic world knowledge remain brittle and representationally distinct, and authoritative contradictory in-context evidence can dramatically reduce belief in some cases (Slocum et al., 20 Oct 2025).

Taken together, these results suggest that superficial belief is best understood as a diagnostic category for belief-like performance that is structured and often useful, yet insufficiently invariant, insufficiently introspectable, or insufficiently integrated with deeper causal structure. In current LLM research, that diagnosis applies to internal truthfulness representations, articulated reasons for choices, and a substantial fraction of alignment behavior. The broader literature indicates that analogous phenomena arise in domain-generalization, Bayesian information processing, and human trust in AI. The central research problem is therefore not merely to detect belief-like behavior, but to determine when such behavior is robust enough to survive harmless variation, scrutiny, and transfer.