Distributional Semantics Tracing (DST)

• DST is an instance-level interpretability framework that constructs semantic networks from layer embeddings to trace causal pathways leading to hallucinations.
• It integrates attention flows, probing classifiers, and gradient attributions to measure semantic drift using metrics like cosine similarity and KL divergence.
• DST empirically links declining contextual coherence, quantified by Distributional Semantics Strength, to rising hallucination rates in language model outputs.

Distributional Semantics Tracing (DST) is a unified, instance-level interpretability framework for LLMs that traces the evolution of high-dimensional contextual embeddings through the Transformer architecture. DST defines “meaning” as a token’s embedding in context and constructs causal graphs across layers to expose the internal dynamics leading to factually incorrect generations (hallucinations). By combining attention analysis, probing classifiers, and gradient-based attribution, DST yields a mechanistic account of model reasoning, quantifies competition between associative and contextual pathways, and empirically connects semantic coherence with hallucination rates (Bhatia et al., 7 Oct 2025).

1. Formal Framework and Mathematical Foundations

DST considers a tokenized input sequence $\mathrm{Input} = [x_1, \dots, x_n]$ and, for each Transformer layer $\ell$, computes the contextual embedding matrix $h^{(\ell)} \in \mathbb{R}^{n \times d}$ as

$$h^{(\ell)} = \mathrm{TransformerLayer}^{(\ell)} \left(h^{(\ell-1)}\right), \quad h^{(0)} = \mathrm{Embed}(x).$$

The core construction at each layer is a semantic network $G^{(\ell)}$, whose nodes are concepts (token clusters/features), and where a directed edge $\Omega^{(\ell)}(A \to B)$ encodes the causal influence of concept $A$ on $B$. DST populates $G^{(\ell)}$ by integrating:

• Attention flows: quantifying path-specific dependencies.
• Probing classifier outputs: exposing semantic content at each node.
• Gradient-based attributions: measuring local sensitivity.

For semantic drift, DST employs two metrics:

• Cosine similarity: $$\cos(h_i^{(\ell)}, r_C) = \frac{\langle h_i^{(\ell)}, r_C \rangle}{\|h_i^{(\ell)}\| \|r_C\|}$$, comparing a token’s embedding to a reference for concept $\ell$0.
• KL divergence: $\ell$1, capturing output shift after patching token $\ell$2 at layer $\ell$3.

Edge weights aggregate these signals: $\ell$4 with $\ell$5 as hyperparameters.

The layerwise networks are merged into a directed acyclic explanation graph mapping inputs to model outputs, identifying dominant reasoning pathways.

2. Algorithmic Pipeline and Application

The DST pipeline comprises the following steps:

1. Causal Path Tracing: Identify critical attention heads and MLP components contributing to output logits.
2. Layerwise Interventions (“Patchscopes”): Replace $\ell$6 with a factual reference to quantify representational drift ($\ell$7 or $\ell$8).
3. Subsequence Tracing: Trace failure back to initiating tokens by correlating $\ell$9 with reference “failure” embeddings.
4. Semantic Network Construction: Assemble $h^{(\ell)} \in \mathbb{R}^{n \times d}$0 using the above signals and compute edge weights $h^{(\ell)} \in \mathbb{R}^{n \times d}$1.
5. Aggregation: Layerwise graphs are integrated into a global explanation mapping the influence of input concepts to outputs.

Practical steps for deploying DST involve extracting activations ($h^{(\ell)} \in \mathbb{R}^{n \times d}$2), attention matrices, and logits at each layer, applying patching, computing cosine and KL divergence scores, constructing $h^{(\ell)} \in \mathbb{R}^{n \times d}$3, and identifying characteristic structural failures.

3. Hallucination Dynamics and the Commitment Layer

DST exposes a cascade of semantic failures culminating in hallucination through three empirically defined stages:

• Prediction Onset ($h^{(\ell)} \in \mathbb{R}^{n \times d}$4): First consistent emergence of a spurious pathway.
• Semantic Inversion ($h^{(\ell)} \in \mathbb{R}^{n \times d}$5): First layer where causal strength of the incorrect pathway $h^{(\ell)} \in \mathbb{R}^{n \times d}$6 exceeds that of the correct one.
• Commitment Layer ($h^{(\ell)} \in \mathbb{R}^{n \times d}$7): The earliest layer $h^{(\ell)} \in \mathbb{R}^{n \times d}$8 such that for all subsequent $h^{(\ell)} \in \mathbb{R}^{n \times d}$9,

$$h^{(\ell)} = \mathrm{TransformerLayer}^{(\ell)} \left(h^{(\ell-1)}\right), \quad h^{(0)} = \mathrm{Embed}(x).$$0

where DSS is the Distributional Semantics Strength (contextual coherence) and $$h^{(\ell)} = \mathrm{TransformerLayer}^{(\ell)} \left(h^{(\ell-1)}\right), \quad h^{(0)} = \mathrm{Embed}(x).$$1 is a threshold (e.g., $$h^{(\ell)} = \mathrm{TransformerLayer}^{(\ell)} \left(h^{(\ell-1)}\right), \quad h^{(0)} = \mathrm{Embed}(x).$$2). At this point, internal representations irreversibly lose the correct semantic pathway.

This progression formally captures where and how commitments to hallucinated outputs become inevitable within the Transformer’s computation.

4. Dual-Process Decomposition: Associative vs. Contextual Pathways

DST reveals that Transformer decisions arise from competition between (i) a fast, associative (“System 1”) pathway, and (ii) a slow, compositional (“System 2”) pathway.

• Associative Pathway: Activated by token co-occurrence statistics, attributed to MLP key-value memory mechanisms, and characterized by low-context, high-saliency output responses.
• Contextual Pathway: Constructed through dynamic self-attention, integrating tokens into context-specific, compositional chains.

DST quantifies contextual dominance via the metric Distributional Semantics Strength (DSS): $$h^{(\ell)} = \mathrm{TransformerLayer}^{(\ell)} \left(h^{(\ell-1)}\right), \quad h^{(0)} = \mathrm{Embed}(x).$$3 where $$h^{(\ell)} = \mathrm{TransformerLayer}^{(\ell)} \left(h^{(\ell-1)}\right), \quad h^{(0)} = \mathrm{Embed}(x).$$4 are contextually correct, $$h^{(\ell)} = \mathrm{TransformerLayer}^{(\ell)} \left(h^{(\ell-1)}\right), \quad h^{(0)} = \mathrm{Embed}(x).$$5 all active pathways. DSS near 1 implies semantic integrity; DSS near 0 denotes contextual collapse and associative override.

Coherence $$h^{(\ell)} = \mathrm{TransformerLayer}^{(\ell)} \left(h^{(\ell-1)}\right), \quad h^{(0)} = \mathrm{Embed}(x).$$6 is identified with DSS: $$h^{(\ell)} = \mathrm{TransformerLayer}^{(\ell)} \left(h^{(\ell-1)}\right), \quad h^{(0)} = \mathrm{Embed}(x).$$7 DST thus enables layerwise, quantitative decomposition of the system’s reasoning modes.

5. Empirical Correlation with Hallucination Rates

DST establishes a direct empirical link between contextual coherence and propensity to hallucinate. For each model-prompt pair:

• Compute mean DSS, $$h^{(\ell)} = \mathrm{TransformerLayer}^{(\ell)} \left(h^{(\ell-1)}\right), \quad h^{(0)} = \mathrm{Embed}(x).$$8, across prompt batches.
• Hallucination Rate: fraction of generations with factually incorrect output, as determined by benchmark verifiers (HALoGEN, Racing Thoughts).

Pearson correlation over these measurements yields

$$h^{(\ell)} = \mathrm{TransformerLayer}^{(\ell)} \left(h^{(\ell-1)}\right), \quad h^{(0)} = \mathrm{Embed}(x).$$9

indicating that declining contextual coherence robustly predicts hallucination rates. The linear fit’s significance was confirmed via OLS regression with reported $G^{(\ell)}$0-values and confidence intervals (Bhatia et al., 7 Oct 2025).

6. Mechanistic Case Study: Reasoning Shortcut Hijack

DST’s layerwise tracing exposes hallmark failures such as the “Reasoning Shortcut Hijack:”

1. In a prompt (“Elon Ganor is a researcher at a quantum‐tech firm”), contextually correct (Elon Ganor → quantum-tech → founder) and spurious associative (Elon + tech → Musk) pathways are simultaneously active at early layers ($G^{(\ell)}$1).
2. At the semantic inversion layer ($G^{(\ell)}$2), the associative pathway overtakes.
3. By the commitment layer ($G^{(\ell)}$3), DSS approaches zero; only the spurious chain is active.
4. The model outputs “Musk” (hallucinated), while DST’s causal graph confirms the absence of a valid contextual pathway to the correct answer.
5. DST flags the hallucination, localizes root causes, and dissects the computational bifurcation driving the erroneous output.

7. Applications and Implications

DST enables granular, local explanations of LLM outputs, provides a principled method to diagnose and predict hallucinations, and offers actionable guidance for interventions. Application requires instrumenting the model to extract activations and attention, running causal tracing and patching, building semantic graphs, computing DSS, and visualizing pathway dynamics. DST further delineates failure patterns including Reasoning Shortcut Hijacks and Analogical Collapse, facilitating targeted architectural and dataset refinements.

A plausible implication is that model improvements should target reinforcement of the contextual pathway’s dominance, as quantified by DSS, especially beyond $G^{(\ell)}$4. DST thus offers a rigorous analytic tool for mechanistically interpreting, benchmarking, and ultimately mitigating LLM hallucinations (Bhatia et al., 7 Oct 2025).