From Signals to Transfer: A Factorised Study of Probe-Based Uncertainty Estimation in Large Language Models

Abstract: Probe-based uncertainty estimation (UE) has emerged as a prominent approach to detect hallucinations in LLMs by learning uncertainty from internal model signals. Yet, recent methods vary simultaneously across feature design, training data construction, and evaluation setting, obscuring what actually drives performance. To address this issue, we propose a factorised study of probe-based UE under matched conditions. Our results show that raw hidden states and attention features are difficult to outperform in-domain. However, under distribution shift, structured and compressed features are more robust, suggesting that in-domain performance alone is insufficient to measure progress. Furthermore, prompting and label construction significantly affect probe behaviour. Building on these best-practice findings, we train benchmark-based pretrained probes that transfer reasonably well to open-ended factual generation, providing a stable off-the-shelf baseline. Our work encourages more deployment-oriented evaluation of probe-based uncertainty estimators. The code repository is available at https://github.com/ponhvoan/ProbeUE.

• The paper demonstrates that simple hidden state and attention-derived features paired with linear probes achieve competitive in-domain uncertainty estimation.
• It shows that structured and compressed features significantly enhance probe transferability and robustness under distribution shifts.
• The study reveals that concise prompt design and high-fidelity semantic labels are crucial for effective and reliable uncertainty evaluation.

Factorised Analysis of Probe-Based Uncertainty Estimation in LLMs

Motivation and Research Objectives

Probe-based uncertainty estimation (UE) has become central in addressing hallucination detection for LLMs. Despite prolific engineering in feature construction, probe architecture, and supervision protocols, confounding across these factors has obstructed clear attribution of performance gains and hampered generalisation analysis. The paper "From Signals to Transfer: A Factorised Study of Probe-Based Uncertainty Estimation in LLMs" (2606.27679) systematically deconstructs probe-based UE across feature representations, training data construction, and evaluation/transfer settings, challenging prevailing assumptions about complexity and transferability. The study identifies robust, practical configurations that support reliable deployment, especially for open-ended factual generation settings.

In-Domain Performance: Feature Representation and Probe Architecture

A core finding is that simple hidden state and attention-derived features—such as mean/last embeddings and Lookback Ratio—are highly competitive for in-domain uncertainty estimation, especially when paired with lightweight linear probes. The study demonstrates that increasingly elaborate feature engineering or probe architecture yields diminishing returns unless the underlying signal type is weak. Moreover, fusion of heterogeneous signals (e.g., combining embeddings and logit-based MSP/Entropy) can modestly enhance discriminability, but concatenation of similar hidden state features does not consistently improve robustness.

Figure 1: Average AUROC quantifies feature discriminability for probe-based UE across seven benchmarks, revealing the strength of simple hidden state and attention signals.

The analysis further reveals that even limited supervision (128–256 examples) suffices to attain saturation in probe performance, indicating the high accessibility of truth-aligned information in LLM internal states.

Figure 2: Probe architecture impacts AUROC marginally for strong feature types, but MLP/CNN probes only benefit weak features.

Figure 3: Probe AUROC rapidly improves with increasing training examples, plateauing at moderate supervision sizes.

Data Construction: Prompting and Label Fidelity

The paper provides a rigorous examination of data construction choices. Reasoning induction via CoT prompt diminishes probe discriminability across multiple feature types, contradicting expectations that reasoning traces inherently enhance self-verification. Concise, direct answer prompts yield higher AUROC scores, especially for low-dimensional signals.

Figure 4: Reasoning prompts degrade probe performance, especially for probability and internal-variance features.

For annotation, LLM-as-a-judge labels (Gemini-3.1-Flash-Lite, GPT-5.4-Mini) achieve much higher agreement with human annotators than lexical scorers (Rouge, AlignScore), supporting semantic correctness alignment as critical for meaningful probe supervision.

Figure 5: LLM-generated correctness labels result in higher probe performance vs. lexical matching labels.

Transfer Analysis: Robustness Under Distribution Shift

Probe brittleness under distribution shift is empirically validated. Benchmark-to-benchmark transfer performance drops sharply for raw hidden state embeddings, which encode task-specific artifacts, while structured features such as Lookback Ratio, Internal Variance, and Layer Top-$m$ Prob. retain higher AUROC across out-of-domain test sets.

Figure 6: Structured/compressed features demonstrate superior average AUROC in benchmark-transfer (OOD) evaluation, narrowing the in-domain advantage of simpler raw embeddings.

The calibration analysis exposes a disconnect between discriminability (AUROC) and calibration (ECE), where structured low-dimensional features maintain stability under shift.

Pretrained Probes for Open-Ended Factual Generation

Extending probe-based UE to long-form, open-ended factual generation, benchmark-pretrained probes—trained on composite benchmark data with best-practice protocols—achieve robust transfer. Structured features again outperform task-specific supervised baselines trained on in-domain samples, substantiating the value of pretraining and feature compression for practical deployment. Ensembles of probes provide mixed benefits, suggesting future directions in transfer robustness.

Practical and Theoretical Implications

The study invalidates the assumption that complex feature engineering or higher-model capacity automatically improves probe-based UE. Instead, it prescribes a pragmatic recipe—lightweight linear probes coupled with transfer-robust, structured features and high-fidelity semantic supervision. This approach optimises both discriminability and calibration in practical (deployment-focused) settings, substantially reducing the supervision burden. Theoretical implications include refined understanding of how internal signals encode truth-related geometry and uncertainty, and how these signals generalise (or fail to generalise) under distributional shift. The findings will inform future work in probe-guided mitigation, automated uncertainty calibration, and adaptation strategies in low-data settings.

Speculation on Future Developments

Further research should focus on broadening coverage to unconstrained dialogues and multi-turn settings, extending probe adaptation protocols to instruction-tuned LLMs, and developing advanced mitigation strategies anchored by uncertainty probes. Robust calibration and signal alignment under extreme distributional shifts remain open challenges.

Conclusion

This paper rigorously factorises probe-based UE in LLMs. It finds that simple internal features and linear probes suffice for in-domain hallucination detection, but only structured/compressed features generalise robustly to new domains and long-form factual generation. Pretrained probes deliver practical reliability without target-task data, provided semantic labels are used. The analysis calls for a shift from increasingly complex probe construction to deployment-oriented configurations, prioritising transfer stability and label fidelity. This reorientation will facilitate improved hallucination detection and uncertainty estimation across real-world LLM applications.