Low-Cost Black-Box Detection of LLM Hallucinations via Dynamical System Prediction

Abstract: LLMs frequently generate plausible but non-factual content, a phenomenon known as hallucination. While existing detection methods typically rely on computationally expensive sampling-based consistency checks or external knowledge retrieval, we propose a new method that treats the LLM as a black-box dynamical system. By projecting LLM responses into a high-dimensional manifold via an embedding model, we characterize the resulting vector sequences as observable realizations of the model's latent state-space dynamics. Leveraging Koopman operator theory, we fit the transition operators for both factual and hallucinated regimes and define a differential residual score based on their respective prediction errors. To accommodate varying user requirements and domain-specific sensitivities, we introduce a preference-aware calibration mechanism that optimizes the classification threshold based on a small set of demonstrations. This approach enables low-cost hallucination detection in a single-sample pass, avoiding the need for secondary sampling or external grounding. Extensive testing across three data benchmarks demonstrates that our method achieves state-of-the-art performance with reduced resource overhead.

• The paper introduces a black-box dynamical systems framework that uses Koopman operator theory to predict token embedding trajectories and classify hallucinations via a differential residual score.
• The DS method achieves state-of-the-art performance, reaching up to 99.3% accuracy and nearly perfect AUC in single-pass detection on benchmarks like HaluEval.
• The approach enables user-adaptable threshold calibration and demonstrates cross-embedding generalization, making it effective for real-time, resource-light applications.

Low-Cost Black-Box Detection of LLM Hallucinations via Dynamical System Prediction

Introduction and Motivation

The endemic issue of hallucinations—plausible but incorrect outputs—in LLMs remains a bottleneck for their reliable deployment in sensitive domains. The prevailing detection approaches demand either high-privilege access to model internals or are reliant on costly sampling or external knowledge retrieval. This paper introduces a fundamentally different paradigm: treating LLM generation as a black-box dynamical system and employing operator-theoretic methods to detect hallucinations from a single output pass, with no need for token-level confidence scores or external ground truth.

Methodology: Dynamical Systems Approach to Hallucination Detection

The core methodological contribution is a dynamical systems (DS) framework that views the evolution of token embeddings in generated text as trajectories on a high-dimensional manifold. Drawing from Koopman operator theory, the authors construct two distinct dynamical models: one fitted to embedding trajectories arising from factual responses and another from hallucinated responses, each inferring a linear operator that best predicts the temporal evolution within its respective class.

Classification hinges on the differential residual score $\Delta \mathcal{E}$, defined as the difference in predictive error between these two models for a given LLM-generated passage. Intuitively, a factual sentence's embedding trajectory will be better predicted by the factual Koopman model, and vice versa for hallucinations. The method operates in two offline/online phases:

• Phase 1 (Model Fitting): Koopman operators are fitted to annotated accurate and hallucinated corpora by embedding token sequences and constructing lifted observable spaces, using Extended Dynamic Mode Decomposition (EDMD).
• Phase 2 (Inference): A new candidate sequence is embedded and scored under both Koopman models. The sign and magnitude of $\Delta \mathcal{E}$ inform hallucination classification via a tunable threshold $\eta$.

Figure 2: The bipartite learning and inference procedure for hallucination detection using parallel Koopman-based models on factual and hallucinated trajectories.

The decision threshold itself is calibrated in a user-centric manner. Through a handful of user-provided demo samples (reflecting their desired tolerance to minor inaccuracies), the method adapts $\eta$ to align with practical deployment contexts, offering a tailored precision–recall trade-off.

Figure 4: Calibration pipeline for user-centric tuning of the hallucination detection threshold based on explicit preference demonstrations.

Experimental Evaluation

Datasets and Embedding Models

The authors conduct thorough experimentation across three benchmarks covering diverse granularity and domains: WikiBio (sentence-level factual drift), HaluEval (summarization-level accuracy), and FELM (domain-specialized reasoning). Five state-of-the-art embedding models with varying architectural features and capacity (from 30M to 8B parameters) are tested to reveal dependence of DS performance on embedding space quality.

Main Findings

Single-Pass State-of-the-Art Detection

A standout empirical result is that the DS method consistently matches or exceeds prior SOTA black-box detectors, including SelfCheckGPT and various LLM-based and heuristic alternatives, while operating in a one-shot regime and eliminating secondary sampling or retrieval (2605.05134). For example, the method using Llama-3 embeddings reaches 99.3% accuracy and near-perfect AUC for hallucination detection on HaluEval with a single model pass.

Sentence-Level Detection and Class Sensitivity

Results on WikiBio demonstrate robust sentence-level performance, with the DS approach (Llama-3 embedding) outperforming SelfCheckGPT in AUC-PR for non-factual sentences. Notably, the magnitude of differential residuals shows graded sensitivity to the severity of hallucination (major vs. minor errors), enabling calibratable detection sensitivity.

Sequence Length and Embedding Quality

Detection accuracy scales with output sequence length: longer trajectories yield improved separation of factual and hallucinated manifolds in embedding space, especially for smaller embedding models. High-capacity embeddings (e.g., Llama-3) deliver almost saturating performance even at shorter lengths.

Cross-Embedding Generalization

Remarkably, although Koopman models are embedding-specific, there is measurable cross-embedding generalization: operators fit on one embedding space can, with non-trivial accuracy, detect hallucinations on trajectories embedded with a different model. This suggests underlying statistical similarities in semantic dynamics across modern embedding spaces.

User-Adaptable Thresholding

User studies and calibration experiments demonstrate that the system’s detection strictness can be seamlessly adapted to application-specific risk tolerance by adjusting $\eta$ on a handful of labeled samples.

Theoretical and Practical Implications

The dynamical system framing posits that hallucinated and factual texts evolve along distinguishable manifolds in semantic space, and that output token trajectories carry rich, model-agnostic indicators of factuality. This recasts traditional “static” text classification and consistency-checking as a temporal inference problem, opening a new axis of model behavior analysis.

Practically, the approach’s API-agnostic, retrieval-free, and resource-light profile addresses key industry pain points: it is suitable for closed-source LLMs and real-time deployment scenarios where latency and cost are paramount. The operator-theoretic machinery establishes a modular bridge for future extension to white-box analysis, knowledge-grounded correction, or ensemble manifold regularization.

Limitations and Future Directions

Despite its strengths, the DS framework does not address correction or mitigation of hallucinations; it is singularly diagnostic. The black-box approximation ignores LLM stochasticity, though robustness to such noise is argued. Further, while cross-embedding generalization is observed, it is not universal, and the embedding space selection remains an important variable.

A promising direction is theoretical exploration of the geometric or topological invariants of embedding dynamics associated with factuality, and integrating DS-based detectors with prompt-engineering or model-steering frameworks for joint detection and mitigation.

Conclusion

This work reframes LLM hallucination detection as a problem in dynamical systems identification, leveraging operator-theoretic methods for resource-efficient, black-box, and user-adaptable detection. Empirical results establish strong performance with minimal computational overhead. This methodology expands the toolkit for practical LLM safety and offers fertile ground for the analysis of high-dimensional generative behavior and the automated safeguarding of AI-generated content.