---
title: 'ICR Probe: Residual Dynamics in Hallucination Detection'
url: https://www.emergentmind.com/topics/icr-probe
type: topic
---

# ICR Probe: Residual Dynamics in Hallucination Detection

ICR Probe is a hallucination detection method for large language models that treats hallucination detection as a problem of tracking the cross-layer evolution of hidden states rather than classifying static representations. It is built around the ICR Score, short for “Information Contribution to Residual Stream,” which quantifies how strongly the residual-stream update at a given layer aligns with the model’s attention pattern. The method uses these layerwise dynamics as a compact feature sequence for a small multilayer perceptron that predicts whether a generated answer is hallucinated, and it is presented as a reference-free, single-generation detector with strong parameter efficiency and an explicitly mechanistic interpretation [2507.16488].

## 1. Definition and problem setting

ICR Probe was introduced to address a limitation in hidden-state-based hallucination detection for decoder-only transformers. Earlier hidden-state approaches often select one or a few layers and treat their activations as static snapshots, whereas ICR Probe is based on the premise that hallucination-related signals are distributed across the depthwise update process of the residual stream [2507.16488].

The method is motivated by a mechanistic distinction between multi-head self-attention and feed-forward networks. In the framework adopted by the paper, multi-head self-attention acts as a contextual signal router that redistributes information already present in token representations, while feed-forward networks act as key–value memory and knowledge injectors that write new information into the residual stream. This makes the hidden-state update process, rather than any isolated hidden state, the primary object of analysis. A central claim of the method is that hallucinated and non-hallucinated generations exhibit systematically different cross-layer patterns of module contribution to hidden-state updates [2507.16488].

The task setting is single-pass hallucination detection. Unlike approaches based on semantic entropy or self-consistency, ICR Probe does not require multiple generations. Unlike retrieval-based verification or external checking, it does not require external evidence. Its input is the hidden-state and attention information already produced during one forward pass of the base model [2507.16488].

## 2. Residual-stream dynamics and the ICR Score

The method starts from the standard residual-stream update at layer $\ell$ for token $i$:
$$
x^\ell_i = x^{\ell-1}_i + \underbrace{\text{MHSA}^\ell(x^{\ell-1}_i)}_{a^\ell_i} + \underbrace{\text{FFN}^\ell(x^{\ell-1}_i + a^\ell_i)}_{m^\ell_i}
$$
and defines the residual-stream update vector
$$
\Delta x^\ell_i = x^\ell_i - x^{\ell-1}_i = a^\ell_i + m^\ell_i.
$$
The update is then compared against the attention pattern of the same token and layer [2507.16488].

For each attention head $h$, the attention logit from token $i$ to token $j$ is
$$
\text{Attn}_{i,j}^{\ell,h} = \frac{(Q_i^h)^T \cdot K_j^h}{\sqrt{d}},
$$
and the paper averages these logits over heads to obtain a token-level attention profile $\text{Attn}_i^\ell$. To construct a comparable representation of the update direction, it projects $\Delta x_i^\ell$ onto the hidden states of all tokens at the same layer:
$$
p_{i,j}^\ell = \frac{(\Delta x^\ell_i)^T \cdot x^\ell_j}{\|x^\ell_j\|}.
$$
After softmax normalization over $j$, this yields a distribution $\text{Proj}_i^\ell$ over context tokens [2507.16488].

The ICR Score is the Jensen–Shannon divergence between these two distributions:
$$
\text{ICR}_i^\ell = \text{JSD}\bigl(\text{Proj}_i^\ell,\text{Attn}_i^\ell\bigr).
$$
Its interpretation is explicitly dynamical. A low ICR Score means that the direction of the residual-stream update is close to the attention pattern, which the paper interprets as an update dominated by attention-mediated routing. A high ICR Score means that the update direction diverges from the attention pattern, which the paper interprets as stronger feed-forward-network dominance, corresponding to parametric knowledge injection rather than direct contextual redistribution [2507.16488].

In practice, the score is computed on the top-$k$ attention positions rather than all tokens. The ablation reported in the paper shows the best AUROC at $k=20$, while using all tokens is worse, indicating that restricting the comparison to the most attended tokens reduces noise [2507.16488].

## 3. Probe architecture and inference pipeline

ICR Probe uses the full matrix of token-layer ICR Scores as an intermediate representation. For an answer with $N$ generated tokens and a model with $L$ layers, this produces an $N \times L$ matrix. The paper then performs token-wise averaging within each layer,
$$
f_\ell = \frac{1}{N}\sum_{i=1}^{N}\text{ICR}_i^\ell,
$$
and forms a compact feature vector
$$
f = [f_1,\dots,f_L] \in \mathbb{R}^L.
$$
This compression is consequential: the detector consumes one scalar per layer rather than thousands of hidden-state coordinates per layer [2507.16488].

The probe itself is a multilayer perceptron with architecture
$$
L \rightarrow 128 \rightarrow 64 \rightarrow 32 \rightarrow 1.
$$
Hidden layers use Leaky ReLU with $\alpha = 0.01$, followed by batch normalization and dropout with $p=0.3$, and the output layer uses a sigmoid to produce a hallucination probability. Training uses binary cross-entropy, Adam with learning rate $5\times 10^{-4}$, ReduceLROnPlateau scheduling, 50 epochs, and batch size 32 [2507.16488].

The parameter count is deliberately small. For typical models with at most 42 layers, the paper reports fewer than 16K parameters for ICR Probe, compared with approximately 110K parameters for SAPLMA. This compactness is a direct consequence of representing hidden-state dynamics by a layerwise ICR trajectory rather than by raw hidden states [2507.16488].

The feature extraction procedure requires access to per-layer hidden states and per-layer attention information. The base LLM remains frozen throughout; only the probe is trained. The method is therefore a lightweight post hoc detector rather than a fine-tuning strategy for the underlying language model [2507.16488].

## 4. Empirical evaluation and performance

The paper evaluates ICR Probe on HaluEval, SQuAD, HotpotQA, and TriviaQA using Llama-3-8B-Instruct, Qwen2.5-7B-Instruct, and Gemma-2-9B-it as backbones. For each dataset, 10,000 instances are sampled and split 80/20 into train and test, and AUROC is used as the principal metric [2507.16488].

Across almost all model–dataset pairs, the method is reported as the best or tied best. For Gemma-2-9B-it, the reported AUROCs are 0.8436 on HaluEval, 0.8142 on SQuAD, 0.8409 on HotpotQA, and 0.8001 on TriviaQA. On the same set of tasks, SAPLMA records 0.8101, 0.7175, and 0.8193 on HaluEval, SQuAD, and HotpotQA respectively, while SEP is reported at 0.6429 on HaluEval and 0.7834 on TriviaQA [2507.16488].

Single-layer ICR Scores are already predictive. For Qwen2.5 on HaluEval, the paper reports AUROC greater than 0.7 for 10 layers and a peak AUROC of about 0.769 at layer 11. Comparable peak single-layer AUROCs are reported for Gemma-2 at about 0.731 and for Llama-3 at about 0.745. The probe improves on this by aggregating cross-layer structure rather than relying on a single best layer [2507.16488].

The method is also evaluated across model scales using Qwen2.5-3B and Qwen2.5-14B. The paper reports that ICR Probe is best on every dataset for both models, with average AUROC gains over SAPLMA of about 3.8–3.9 points and over SEP of about 9–11 points. This supports the claim that the ICR feature sequence is stable across model sizes rather than narrowly tuned to one scale [2507.16488].

Cross-dataset generalization is another reported strength. When trained on one dataset and tested on another, ICR Probe maintains AUROC of at least 0.66. The average in-domain to cross-domain drop is 8.61%, compared with 10.18% for SAPLMA and 11.67% for SEP. The paper interprets this as evidence that ICR captures intrinsic model dynamics that are less dataset-specific than raw hidden-state classifiers [2507.16488].

## 5. Mechanistic interpretation, ablations, and case analyses

A major part of the method’s significance lies in its interpretability. The paper reports a stable layerwise ICR profile across datasets. In Qwen2.5, early layers 0–3 show low ICR, middle layers 4–20 rise toward a peak around layers 10–15, and deeper layers beyond 21 decrease again. This is interpreted as an early attention-dominated stage, a middle stage with stronger feed-forward-network contribution, and a later return to attention-dominated refinement [2507.16488].

Ablation results reinforce this interpretation. When the method is reduced to hidden-state projections only, by replacing the attention distribution with a uniform distribution, AUROC remains strong at about 0.77–0.78. Adding the true attention distribution to form the full ICR Score raises performance further to about 0.80–0.84. This indicates that both the entropy-like structure of update directions and their alignment with attention matter, but the alignment term is not redundant [2507.16488].

Layer-group ablations divide Gemma-2 into early layers 0–13, middle layers 15–28, and deep layers 29–41. Removing middle layers causes the largest performance drop, whereas removing early or deep layers is less damaging. This is consistent with the paper’s claim that middle layers, where feed-forward-network influence peaks, are the most informative region for hallucination detection [2507.16488].

The paper also presents token-level case studies. In a hallucinated answer where the model outputs “18.5 hectares” instead of “100 ha,” the token “hectares” receives a very low non-hallucination probability of about 0.04, while punctuation and whitespace remain near 0.96–0.99. In another hallucinated case, the tokens “Vietnam,” “War,” and “draft” are flagged with low probabilities around 0.17–0.28. In a non-hallucinated answer “The Cree,” the content token “Cree” receives a moderate to high probability of about 0.55. These examples suggest that the probe is particularly sensitive to factual content tokens such as entities, numbers, and units [2507.16488].

## 6. Scope, limitations, and position in the literature

ICR Probe is positioned against both training-free uncertainty measures and prior hidden-state probes. The paper compares it to perplexity, length-normalized entropy, LLM-Check, SAPLMA, and SEP. Relative to probability-based baselines, its distinguishing feature is that it exploits internal residual-stream dynamics rather than only output uncertainty. Relative to hidden-state probes such as SAPLMA, its distinguishing feature is that it models cross-layer evolution instead of selecting a few static layers [2507.16488].

The method’s limitations are explicit. It requires access to hidden states and attention weights or logits, which restricts it to open-weight models or APIs that expose internals. It is a detector rather than a mitigation method; it flags hallucinations but does not alter generation. The evaluation is centered on question-answering tasks, so the method’s behavior on longer-form or more interaction-heavy generation remains an open question. In addition, although cross-dataset generalization is strong, the probe is still trained separately for each backbone model [2507.16488].

Within hallucination detection research, ICR Probe represents a shift from static representation probing to process-level probing. Its central claim is not merely that hidden states contain hallucination signals, but that the way attention and feed-forward modules jointly update the residual stream across depth yields a compact and reliable signature of hallucination risk. In that sense, the method functions simultaneously as a detector and as a mechanistic lens on transformer computation [2507.16488].

Source: https://www.emergentmind.com/topics/icr-probe