ICR Probe: Residual Dynamics in Hallucination Detection

• ICR Probe is a hallucination detection method that tracks cross-layer evolution of hidden states to identify factual inaccuracies in generated outputs.
• It computes the ICR Score using Jensen–Shannon divergence between projected residual updates and attention distributions, reflecting key module contributions.
• The method employs a compact multilayer perceptron architecture, showing robust performance and parameter efficiency across various QA benchmarks.

ICR Probe is a hallucination detection method for LLMs that treats hallucination detection as a problem of tracking the cross-layer evolution of hidden states rather than classifying static representations. It is built around the ICR Score, short for “Information Contribution to Residual Stream,” which quantifies how strongly the residual-stream update at a given layer aligns with the model’s attention pattern. The method uses these layerwise dynamics as a compact feature sequence for a small multilayer perceptron that predicts whether a generated answer is hallucinated, and it is presented as a reference-free, single-generation detector with strong parameter efficiency and an explicitly mechanistic interpretation (Zhang et al., 22 Jul 2025).

1. Definition and problem setting

ICR Probe was introduced to address a limitation in hidden-state-based hallucination detection for decoder-only transformers. Earlier hidden-state approaches often select one or a few layers and treat their activations as static snapshots, whereas ICR Probe is based on the premise that hallucination-related signals are distributed across the depthwise update process of the residual stream (Zhang et al., 22 Jul 2025).

The method is motivated by a mechanistic distinction between multi-head self-attention and feed-forward networks. In the framework adopted by the paper, multi-head self-attention acts as a contextual signal router that redistributes information already present in token representations, while feed-forward networks act as key–value memory and knowledge injectors that write new information into the residual stream. This makes the hidden-state update process, rather than any isolated hidden state, the primary object of analysis. A central claim of the method is that hallucinated and non-hallucinated generations exhibit systematically different cross-layer patterns of module contribution to hidden-state updates (Zhang et al., 22 Jul 2025).

The task setting is single-pass hallucination detection. Unlike approaches based on semantic entropy or self-consistency, ICR Probe does not require multiple generations. Unlike retrieval-based verification or external checking, it does not require external evidence. Its input is the hidden-state and attention information already produced during one forward pass of the base model (Zhang et al., 22 Jul 2025).

2. Residual-stream dynamics and the ICR Score

The method starts from the standard residual-stream update at layer $\ell$ for token $i$:

$$x^\ell_i = x^{\ell-1}_i + \underbrace{\text{MHSA}^\ell(x^{\ell-1}_i)}_{a^\ell_i} + \underbrace{\text{FFN}^\ell(x^{\ell-1}_i + a^\ell_i)}_{m^\ell_i}$$

and defines the residual-stream update vector

$$\Delta x^\ell_i = x^\ell_i - x^{\ell-1}_i = a^\ell_i + m^\ell_i.$$

The update is then compared against the attention pattern of the same token and layer (Zhang et al., 22 Jul 2025).

For each attention head $h$, the attention logit from token $i$ to token $j$ is

$$\text{Attn}_{i,j}^{\ell,h} = \frac{(Q_i^h)^T \cdot K_j^h}{\sqrt{d}},$$

and the paper averages these logits over heads to obtain a token-level attention profile $\text{Attn}_i^\ell$. To construct a comparable representation of the update direction, it projects $\Delta x_i^\ell$ onto the hidden states of all tokens at the same layer:

$i$0

After softmax normalization over $i$1, this yields a distribution $i$2 over context tokens (Zhang et al., 22 Jul 2025).

The ICR Score is the Jensen–Shannon divergence between these two distributions:

$i$3

Its interpretation is explicitly dynamical. A low ICR Score means that the direction of the residual-stream update is close to the attention pattern, which the paper interprets as an update dominated by attention-mediated routing. A high ICR Score means that the update direction diverges from the attention pattern, which the paper interprets as stronger feed-forward-network dominance, corresponding to parametric knowledge injection rather than direct contextual redistribution (Zhang et al., 22 Jul 2025).

In practice, the score is computed on the top-$i$4 attention positions rather than all tokens. The ablation reported in the paper shows the best AUROC at $i$5, while using all tokens is worse, indicating that restricting the comparison to the most attended tokens reduces noise (Zhang et al., 22 Jul 2025).

3. Probe architecture and inference pipeline

ICR Probe uses the full matrix of token-layer ICR Scores as an intermediate representation. For an answer with $i$6 generated tokens and a model with $i$7 layers, this produces an $i$8 matrix. The paper then performs token-wise averaging within each layer,

$i$9

and forms a compact feature vector

$$x^\ell_i = x^{\ell-1}_i + \underbrace{\text{MHSA}^\ell(x^{\ell-1}_i)}_{a^\ell_i} + \underbrace{\text{FFN}^\ell(x^{\ell-1}_i + a^\ell_i)}_{m^\ell_i}$$0

This compression is consequential: the detector consumes one scalar per layer rather than thousands of hidden-state coordinates per layer (Zhang et al., 22 Jul 2025).

The probe itself is a multilayer perceptron with architecture

$$x^\ell_i = x^{\ell-1}_i + \underbrace{\text{MHSA}^\ell(x^{\ell-1}_i)}_{a^\ell_i} + \underbrace{\text{FFN}^\ell(x^{\ell-1}_i + a^\ell_i)}_{m^\ell_i}$$1

Hidden layers use Leaky ReLU with $$x^\ell_i = x^{\ell-1}_i + \underbrace{\text{MHSA}^\ell(x^{\ell-1}_i)}_{a^\ell_i} + \underbrace{\text{FFN}^\ell(x^{\ell-1}_i + a^\ell_i)}_{m^\ell_i}$$2, followed by batch normalization and dropout with $$x^\ell_i = x^{\ell-1}_i + \underbrace{\text{MHSA}^\ell(x^{\ell-1}_i)}_{a^\ell_i} + \underbrace{\text{FFN}^\ell(x^{\ell-1}_i + a^\ell_i)}_{m^\ell_i}$$3, and the output layer uses a sigmoid to produce a hallucination probability. Training uses binary cross-entropy, Adam with learning rate $$x^\ell_i = x^{\ell-1}_i + \underbrace{\text{MHSA}^\ell(x^{\ell-1}_i)}_{a^\ell_i} + \underbrace{\text{FFN}^\ell(x^{\ell-1}_i + a^\ell_i)}_{m^\ell_i}$$4, ReduceLROnPlateau scheduling, 50 epochs, and batch size 32 (Zhang et al., 22 Jul 2025).

The parameter count is deliberately small. For typical models with at most 42 layers, the paper reports fewer than 16K parameters for ICR Probe, compared with approximately 110K parameters for SAPLMA. This compactness is a direct consequence of representing hidden-state dynamics by a layerwise ICR trajectory rather than by raw hidden states (Zhang et al., 22 Jul 2025).

The feature extraction procedure requires access to per-layer hidden states and per-layer attention information. The base LLM remains frozen throughout; only the probe is trained. The method is therefore a lightweight post hoc detector rather than a fine-tuning strategy for the underlying LLM (Zhang et al., 22 Jul 2025).

4. Empirical evaluation and performance

The paper evaluates ICR Probe on HaluEval, SQuAD, HotpotQA, and TriviaQA using Llama-3-8B-Instruct, Qwen2.5-7B-Instruct, and Gemma-2-9B-it as backbones. For each dataset, 10,000 instances are sampled and split 80/20 into train and test, and AUROC is used as the principal metric (Zhang et al., 22 Jul 2025).

Across almost all model–dataset pairs, the method is reported as the best or tied best. For Gemma-2-9B-it, the reported AUROCs are 0.8436 on HaluEval, 0.8142 on SQuAD, 0.8409 on HotpotQA, and 0.8001 on TriviaQA. On the same set of tasks, SAPLMA records 0.8101, 0.7175, and 0.8193 on HaluEval, SQuAD, and HotpotQA respectively, while SEP is reported at 0.6429 on HaluEval and 0.7834 on TriviaQA (Zhang et al., 22 Jul 2025).

Single-layer ICR Scores are already predictive. For Qwen2.5 on HaluEval, the paper reports AUROC greater than 0.7 for 10 layers and a peak AUROC of about 0.769 at layer 11. Comparable peak single-layer AUROCs are reported for Gemma-2 at about 0.731 and for Llama-3 at about 0.745. The probe improves on this by aggregating cross-layer structure rather than relying on a single best layer (Zhang et al., 22 Jul 2025).

The method is also evaluated across model scales using Qwen2.5-3B and Qwen2.5-14B. The paper reports that ICR Probe is best on every dataset for both models, with average AUROC gains over SAPLMA of about 3.8–3.9 points and over SEP of about 9–11 points. This supports the claim that the ICR feature sequence is stable across model sizes rather than narrowly tuned to one scale (Zhang et al., 22 Jul 2025).

Cross-dataset generalization is another reported strength. When trained on one dataset and tested on another, ICR Probe maintains AUROC of at least 0.66. The average in-domain to cross-domain drop is 8.61%, compared with 10.18% for SAPLMA and 11.67% for SEP. The paper interprets this as evidence that ICR captures intrinsic model dynamics that are less dataset-specific than raw hidden-state classifiers (Zhang et al., 22 Jul 2025).

5. Mechanistic interpretation, ablations, and case analyses

A major part of the method’s significance lies in its interpretability. The paper reports a stable layerwise ICR profile across datasets. In Qwen2.5, early layers 0–3 show low ICR, middle layers 4–20 rise toward a peak around layers 10–15, and deeper layers beyond 21 decrease again. This is interpreted as an early attention-dominated stage, a middle stage with stronger feed-forward-network contribution, and a later return to attention-dominated refinement (Zhang et al., 22 Jul 2025).

Ablation results reinforce this interpretation. When the method is reduced to hidden-state projections only, by replacing the attention distribution with a uniform distribution, AUROC remains strong at about 0.77–0.78. Adding the true attention distribution to form the full ICR Score raises performance further to about 0.80–0.84. This indicates that both the entropy-like structure of update directions and their alignment with attention matter, but the alignment term is not redundant (Zhang et al., 22 Jul 2025).

Layer-group ablations divide Gemma-2 into early layers 0–13, middle layers 15–28, and deep layers 29–41. Removing middle layers causes the largest performance drop, whereas removing early or deep layers is less damaging. This is consistent with the paper’s claim that middle layers, where feed-forward-network influence peaks, are the most informative region for hallucination detection (Zhang et al., 22 Jul 2025).

The paper also presents token-level case studies. In a hallucinated answer where the model outputs “18.5 hectares” instead of “100 ha,” the token “hectares” receives a very low non-hallucination probability of about 0.04, while punctuation and whitespace remain near 0.96–0.99. In another hallucinated case, the tokens “Vietnam,” “War,” and “draft” are flagged with low probabilities around 0.17–0.28. In a non-hallucinated answer “The Cree,” the content token “Cree” receives a moderate to high probability of about 0.55. These examples suggest that the probe is particularly sensitive to factual content tokens such as entities, numbers, and units (Zhang et al., 22 Jul 2025).

6. Scope, limitations, and position in the literature

ICR Probe is positioned against both training-free uncertainty measures and prior hidden-state probes. The paper compares it to perplexity, length-normalized entropy, LLM-Check, SAPLMA, and SEP. Relative to probability-based baselines, its distinguishing feature is that it exploits internal residual-stream dynamics rather than only output uncertainty. Relative to hidden-state probes such as SAPLMA, its distinguishing feature is that it models cross-layer evolution instead of selecting a few static layers (Zhang et al., 22 Jul 2025).

The method’s limitations are explicit. It requires access to hidden states and attention weights or logits, which restricts it to open-weight models or APIs that expose internals. It is a detector rather than a mitigation method; it flags hallucinations but does not alter generation. The evaluation is centered on question-answering tasks, so the method’s behavior on longer-form or more interaction-heavy generation remains an open question. In addition, although cross-dataset generalization is strong, the probe is still trained separately for each backbone model (Zhang et al., 22 Jul 2025).

Within hallucination detection research, ICR Probe represents a shift from static representation probing to process-level probing. Its central claim is not merely that hidden states contain hallucination signals, but that the way attention and feed-forward modules jointly update the residual stream across depth yields a compact and reliable signature of hallucination risk. In that sense, the method functions simultaneously as a detector and as a mechanistic lens on transformer computation (Zhang et al., 22 Jul 2025).