HaloScope: Unsupervised Hallucination Detection

• HaloScope is a two-stage framework that detects hallucinations in LLM outputs by analyzing low-dimensional subspaces of internal embeddings.
• It uses a pseudo-labeling approach to split unlabeled data into likely truthful and hallucinated sets, enabling effective supervised classifier training.
• Empirical evaluations show that HaloScope achieves higher AUROC scores than zero-shot, few-shot, and weakly supervised methods across multiple QA benchmarks.

HaloScope denotes a two-stage hallucination detection framework for LLM outputs that leverages the vast pool of unlabeled generations naturally generated by LLMs at inference time, rather than relying on curated or hand-annotated datasets. HaloScope is designed to automatically estimate which samples among unlabeled outputs are likely hallucinated versus truthful by analyzing the subspace structure of internal LLM embeddings. It then trains a supervised classifier on these pseudo-labeled splits, resulting in improved downstream hallucination detection compared to prior zero-shot, few-shot, and weakly supervised alternatives (Du et al., 2024).

1. Problem Setup and Theoretical Assumptions

The core setting addressed by HaloScope is that, for LLM deployments in open-world settings (interactive chatbots, generative QA platforms, etc.), one observes a large collection of unlabeled prompt–generation pairs

$$\mathcal{M} = \{(x^{i}_{\mathrm{in}}, \tilde{x}^i)\}_{i=1}^{N}$$

Each $\tilde{x}^i$ may be a truthful generation (drawn from distribution $\mathbb P_{\mathrm{true}}$) or a hallucinated (untruthful) response ($\mathbb P_{\mathrm{hal}}$). The aggregate distribution is formally modeled by a Huber-contaminated mixture: $$\mathbb P_{\mathrm{unlabeled}} = (1 - \pi)\mathbb P_{\mathrm{true}} + \pi\,\mathbb P_{\mathrm{hal}}, \quad \pi \ll 1$$ Unlike supervised hallucination detection—which requires expensive, large-scale hand-labeled truthful/hallucination pairs—HaloScope operates label-free, inferring likely membership directly from statistics of the generated data (Du et al., 2024).

2. Membership-Estimation via Subspace Analysis

The first stage of HaloScope involves extracting internal representations for each generation, typically by feeding the concatenation of the input and generated answer through the LLM and extracting the embedding of the last token: $$f_i \in \mathbb{R}^d, \;\; F = [f_1^\top; f_2^\top; \ldots; f_N^\top] \in \mathbb{R}^{N \times d}$$ The centralized embedding matrix ($F - \mathbf{1}\mu^\top$) is subjected to a singular value decomposition: $F - \mathbf{1}\mu^\top = U\Sigma V^\top$.

HaloScope posits that a low-dimensional "hallucination subspace," spanned by the top $k$ right singular vectors $v_j$, isolates much of the variation due to hallucinations. Each sample $\tilde{x}^i$0 receives a membership-estimation score

$\tilde{x}^i$1

where $\tilde{x}^i$2 is the $\tilde{x}^i$3-th singular value. Empirically, hallucinated generations tend to have higher $\tilde{x}^i$4, lying farther from the main bulk of "truthful" activations. Thresholding $\tilde{x}^i$5 at some $\tilde{x}^i$6 partitions the data into candidate truthful ($\tilde{x}^i$7) and hallucinated ($\tilde{x}^i$8) sets: $\tilde{x}^i$9 These are treated as noisy labels for classifier training (Du et al., 2024).

3. Supervised Classifier Training from Pseudo-Labels

Given candidate sets $\mathbb P_{\mathrm{true}}$0 (truth) and $\mathbb P_{\mathrm{true}}$1 (hallucination), HaloScope trains a compact multi-layer perceptron $\mathbb P_{\mathrm{true}}$2 on LLM embeddings $\mathbb P_{\mathrm{true}}$3 (typically, the last token in the concatenated input-output), using a standard sigmoid surrogate loss: $\mathbb P_{\mathrm{true}}$4 with $\mathbb P_{\mathrm{true}}$5 for $\mathbb P_{\mathrm{true}}$6 and $\mathbb P_{\mathrm{true}}$7 for $\mathbb P_{\mathrm{true}}$8. The empirical risk minimized is

$\mathbb P_{\mathrm{true}}$9

This binary classifier discriminates between pseudo-labeled truthful and hallucinated generations using only unlabeled deployment data (Du et al., 2024).

4. Empirical Evaluation and Performance

HaloScope has been validated on a battery of four QA benchmarks (TruthfulQA, TriviaQA, TydiQA-GP, and CoQA), using models such as LLaMA-2-chat-7B, OPT-6.7B, and their 13B variants. HaloScope achieves consistently superior performance (as measured by AUROC) relative to competing uncertainty-, consistency-, prompting-, and knowledge-based baselines. For instance:

• On LLaMA-2-7B, TruthfulQA: 78.64% AUROC for HaloScope, compared to 67.95% for an improved CCS* baseline, and ~62% for the best uncertainty/consistency methods.
• On TydiQA-GP: HaloScope achieves 94.04% AUROC on LLaMA-2-7B.
• Larger models (e.g., OPT-13B) see further gains: 82.41% AUROC on TruthfulQA (Du et al., 2024).

Ablation studies in the original paper demonstrate:

• Generalization of the learned subspace between datasets (e.g., TriviaQA to TruthfulQA: only a 2% AUROC drop)
• Best separation using intermediate transformer layers (layers 8–14)
• Robustness to unlabeled dataset size down to $\mathbb P_{\mathrm{hal}}$0
• Optimal subspace dimension is moderate ($\mathbb P_{\mathrm{hal}}$1)
• Direct projection onto the subspace is inferior to the trained classifier stage

HaloScope's test-time computational cost is dominated by embedding extraction and small MLP evaluation per sequence, making it practical for integration into LLM inference pipelines.

5. Practical Considerations, Flexibility, and Limitations

Scalability: The approach uses only one generation per prompt and a single SVD on the collected embedding matrix, with no need for expensive sampling or external annotation.

Data Volume: While performance scales with the number of unlabeled samples, even a few hundred generations suffice to realize substantial detection improvements.

Domain Shift: Detection accuracy may degrade if the distribution of test prompts diverges substantially from the original unlabeled data used to derive the hallucination subspace.

Future Extensions: Incorporation of robust learning methods, domain adaptation for distributional shifts, and iterative refinement of the hallucination subspace are directly suggested as possible future lines of research.

Comparison to Full Supervision: HaloScope approaches the supervised oracle upper bound (81.04% AUROC on TruthfulQA) with no need for any labeled data (Du et al., 2024).

6. Algorithmic Workflow

The HaloScope workflow is systematically as follows:

1. Collect unlabeled prompt–generation pairs from natural LLM deployments.
2. Embed: Extract token embeddings (e.g., last-token) via the underlying LLM.
3. Center and decompose the embedding matrix via SVD.
4. Score: Compute the membership-estimation score $\mathbb P_{\mathrm{hal}}$2.
5. Threshold to define candidate truthful and hallucinated subsets.
6. Train classifier from these subsets on top of frozen LLM representations.
7. Inference: For new data, output a truthfulness score via the trained classifier and designate hallucinations via thresholding (Du et al., 2024).

This two-stage structure (subspace mining, then pseudo-supervision) is the defining operational feature of HaloScope.

7. Significance and Implications

HaloScope constitutes a principled, computationally efficient strategy for hallucination detection in LLMs without requiring labeled data or external feedback. Its substantial performance advantage over prior approaches is robust across model sizes, task domains, and benchmark composition. The method's key insight is the identification of a low-dimensional "hallucination subspace" in LLM activation space, and the ability to bootstrap a competitive truthfulness detector from pseudo-labeled splits derived therefrom. This approach is practical for real-world deployments, particularly when human annotation is infeasible or prohibitive (Du et al., 2024). A direct implication is that HaloScope methodology enables trustworthy deployment of LLM-based applications by leveraging only the statistics of unlabeled, in-the-wild generated data.