HAD: Hallucination Detection in LLMs

• The paper introduces a reference-free, token-level detection method that flags hallucinated tokens in LLM outputs using variance from multiple stochastic samples.
• The method employs N stochastic forward passes with an empirically tuned threshold, balancing precision and recall for accurate detection.
• Empirical results demonstrate that larger, instruction-tuned LLMs exhibit lower hallucination rates, enhancing output reliability and model grounding.

LLMs are capable of generating fluent, convincing, but sometimes factually incorrect or unsupported content—an effect termed hallucination. The development and deployment of HAllucination Detection (HAD) models is crucial for ensuring the reliability of LLMs in knowledge-sensitive environments. HAD frameworks enable token-level, sentence-level, or span-level identification of hallucinated content, offering both diagnostic tools for model analysis and foundations for automated mitigation and correction.

1. Core Principles of Token-Level Hallucination Detection

A foundational advance in HAD is the introduction of unsupervised, reference-free frameworks that operate at token granularity. The approach introduced by Kumar (Kumar, 5 Jul 2025) proposes leveraging the variance of token log-probabilities across multiple stochastic generations as an intrinsic measure of model uncertainty. The key observation is that when an LLM hallucinates—i.e., generates content that lacks contextual or factual grounding—it exhibits instability in its token-level probability assignments. Thus, hallucinated tokens consistently display higher variance in their assigned log-probabilities under repeated stochastic sampling.

For a given prompt $x$, the procedure is as follows. $N$ stochastic forward passes (sampled with identical decoding hyperparameters) are run, resulting in $N$ outputs $y^{(1)}, \dots, y^{(N)}$. For a token position $t$, the token log-probabilities $$\ell_{t,i} = \log p(y_t^{(i)} \mid y_{<t}^{(i)}, x)$$ are extracted for each sample $i$, and the mean $\mu_t$ and variance $\operatorname{Var}(\ell_t)$ are computed: $$\mu_t = \frac{1}{N}\sum_{i=1}^N \ell_{t,i}, \quad \operatorname{Var}(\ell_t) = \frac{1}{N}\sum_{i=1}^N \ell_{t,i}^{2} - \left(\frac{1}{N}\sum_{i=1}^N \ell_{t,i}\right)^2.$$ A token is flagged as hallucinated if $N$0, with $N$1 chosen to optimize token-level F1 on a held-out validation set.

This HAD method is architecturally agnostic and reference-free, requiring no gold labels. It can be integrated into real-time generation pipelines, supporting both proactive suppression and post-hoc analysis.

2. Practical Implementation and Threshold Selection

Implementing HAD involves generating $N$2 stochastic outputs per input (e.g., $N$3, with default sampling parameters such as temperature $N$4 and top-$N$5). Variance schedules are computed in a fully batched inference pipeline, leveraging half-precision computation to minimize overhead and maximize throughput. The framework is designed to be lightweight: for $N$6, additional compute is negligible on modern hardware, enabling both batch-level and streaming mode operation (Kumar, 5 Jul 2025).

Threshold $N$7 is determined empirically: lower thresholds increase recall (more tokens flagged as hallucinated but at the expense of increased false positives), while higher thresholds increase precision but may under-detect subtle hallucinations. Ablations demonstrate that the method is robust to reasonable choices of $N$8 (in the range 0.4–0.6) and that threshold tuning on a small validation set is sufficient to preserve reliability rankings across model scales.

3. Empirical Results and Model Scalability

Quantitative results on 100 unanswerable SQuAD v2 prompts (context truncated to 300 characters) indicate a clear scaling law: the rate of detected hallucinated tokens decreases as model size increases. For example, GPT-Neo 125M has 72.42% of tokens flagged as hallucinated, Falcon 1B at 64.75%, and Mistral 7B at 26.75% using $N$9 and $N$0. Diagnostics with kernel density estimation, cumulative distribution functions, and token-wise heatmaps reveal pronounced variance spikes in smaller models and substantiate that larger, instruction-tuned LLMs produce more reliably grounded outputs (Kumar, 5 Jul 2025).

Qualitative analysis demonstrates that high-variance tokens align with unsupported continuations (e.g., “venture”, “ad”), while larger models maintain low-variance profiles, even in later positions within sampled outputs.

4. Strengths, Limitations, and Operational Guidance

Strengths

• Token-Level Precision: Unlike sentence-level or document-level detectors, this HAD framework enables precise localization of hallucinated content, supporting actionable diagnostics.
• Reference-Free and Model-Agnostic: No externally supplied references or task-specific classifiers required; the framework generalizes across architectures and domains.
• Lightweight and Real-Time Capable: No parameter updates or retraining required; the method is compatible with both real-time filtering and offline auditing.
• Interpretability: Variance profiles, heatmaps, and CDFs provide intuitive signals for model debugging and error analysis.

Limitations

• Limited Sensitivity in Deterministic Outputs: Highly deterministic (low-temperature) or syntactically constrained tasks yield low variance, masking hallucinations.
• Hyperparameter and Threshold Tuning: Careful selection of sampling parameters and $N$1 is necessary for optimal performance and cross-model comparability.
• Task and Scale Dependence: New validations may be needed for each model/task pair due to varying baseline variances and generative entropy.

Best Practices

• Use consistent decoding parameters and prompt formats to ensure comparability.
• Tune $N$2 using a small, representative validation set with available annotations or via gold-free heuristics.
• Log full token-level variance maps to support heatmap-based qualitative analysis.
• Employ real-time filtering strategies during inference: re-sample or suppress output tokens that exceed the fitted variance threshold.

5. Extensions and Broader Applications

Potential extensions of HAD include application to abstractive summarization and open-domain QA datasets such as XSum and TriviaQA, systematic auditing in retrieval-augmented generation pipelines, and adoption in multilingual or multimodal contexts via joint token variance computation. Integrating token-level variance detectors into fine-tuning can further stabilize model predictions by penalizing unstable outputs (Kumar, 5 Jul 2025).

Future work could combine high-variance token signals with lightweight retrieval or entity-linking modules, triggering targeted external verification when model-internal uncertainty crosses a threshold. This hybrid approach opens the way to robust, scalable, and interpretable fact-checking systems in production LLM deployments.

---

References:

• Kumar, A., "Token Level Hallucination Detection via Variance in LLMs," (Kumar, 5 Jul 2025) (2025-07-05) (Kumar, 5 Jul 2025).