Detecting AI Hallucinations in Finance: An Information-Theoretic Method Cuts Hallucination Rate by 92% (2512.03107v1)

Abstract: LLMs produce fluent but unsupported answers - hallucinations - limiting safe deployment in high-stakes domains. We propose ECLIPSE, a framework that treats hallucination as a mismatch between a model's semantic entropy and the capacity of available evidence. We combine entropy estimation via multi-sample clustering with a novel perplexity decomposition that measures how models use retrieved evidence. We prove that under mild conditions, the resulting entropy-capacity objective is strictly convex with a unique stable optimum. We evaluate on a controlled financial question answering dataset with GPT-3.5-turbo (n=200 balanced samples with synthetic hallucinations), where ECLIPSE achieves ROC AUC of 0.89 and average precision of 0.90, substantially outperforming a semantic entropy-only baseline (AUC 0.50). A controlled ablation with Claude-3-Haiku, which lacks token-level log probabilities, shows AUC dropping to 0.59 with coefficient magnitudes decreasing by 95% - demonstrating that ECLIPSE is a logprob-native mechanism whose effectiveness depends on calibrated token-level uncertainties. The perplexity decomposition features exhibit the largest learned coefficients, confirming that evidence utilization is central to hallucination detection. We position this work as a controlled mechanism study; broader validation across domains and naturally occurring hallucinations remains future work.

• The paper introduces the ECLIPSE framework, achieving a 92% reduction in hallucination rate at 30% coverage by balancing output uncertainty with evidential capacity.
• It leverages token-level log probabilities and perplexity decomposition, outperforming semantic entropy-only methods with an ROC AUC of 0.89.
• The study provides theoretical guarantees via convex optimization and offers interpretability insights vital for high-stakes financial language modeling.

Information-Theoretic Hallucination Detection in Financial LLM QA: The ECLIPSE Framework

Introduction and Motivation

The propensity for LLMs to generate syntactically valid but factually unsupported content—hallucinations—has sharply constrained their utility in high-stakes domains such as finance. Existing detection techniques focusing on semantic entropy or consistency between outputs often lack specificity and struggle to differentiate between honest uncertainty (where evidence is ambiguous or partial) and instances where a model confidently ignores high-quality evidence. This paper proposes ECLIPSE (Entropy–Capacity Logprob-Native Inference for Predicting Spurious Emissions), an information-theoretic framework for hallucination detection that isolates risk as a function of the tension between a model’s output uncertainty and the objectively quantifiable informativeness (capacity) of its evidence.

Theoretical Framework

ECLIPSE formalizes hallucination risk through a joint entropy and capacity model. The paper defines semantic entropy ($H$) via clustering generated answers for factual content and quantifies evidence capacity ($C$) by decomposing answer log-likelihoods to measure the support conferred by evidence passages. The logistic hallucination probability function incorporates the deviation of measured entropy from an evidence-conditioned optimal (preferred) entropy, with additional regularization on evidence capacity.

A principal theoretical result is the proof of strict convexity for the entropy–capacity objective under mild conditions ($\alpha > \lambda a^2/8$), guaranteeing the existence and uniqueness of global optima for predictive risk calibration and laying groundwork for potential control protocols.

Practical Implementation and Feature Engineering

ECLIPSE leverages grey-box access to LLM APIs supporting token-level log probabilities, eschewing white-box approaches reliant on hidden state introspection. The evidential capacity features—particularly perplexity decomposition metrics ($L_Q$, $L_{QE}$, $\Delta L$)—enable fine-grained tracking of evidence utilization. The operational detector is a logistic regression over these features, supervised on a balanced financial QA dataset containing both clean and synthetically hallucinated answers.

Empirical Results

The primary benchmark involves 200 financial QA instances derived from SEC filings and transcripts, with hallucinated responses constructed via systematic numeric, entity, directional, and fabrication perturbations.

ECLIPSE achieves ROC AUC of 0.89 and average precision of 0.90 on held-out folds, starkly outperforming the semantic entropy-only baseline (AUC 0.50). Coefficient analysis shows that features characterizing evidence-based lift ($L_{QE}$, $\Delta L$, ratio) dominate learned weights, matching theoretical predictions. Remarkably, the $p_{\max}$ coefficient is unexpectedly positive, indicating that high model confidence paradoxically increases hallucination risk in this domain.

Figure 1: ROC curves for ECLIPSE and entropy-only baseline, demonstrating dominance of ECLIPSE (AUC = 0.89 vs 0.50).

Ablation studies show incremental AUC improvements as feature groups are added: entropy (0.50), + capacity (0.68), + perplexity decomposition (0.89), quantifying substantive contributions of each component.

Figure 2: Ablation of feature sets illustrating AUC stepwise improvement, culminating in a 78% relative gain over entropy-only detection.

Coverage analysis under selective prediction regimes highlights practical value: ECLIPSE cuts hallucination rate by 92% at 30% coverage (from 43.3% to 3.3%), with consistent gains across coverage levels.

Figure 3: Coverage vs hallucination rate, where ECLIPSE substantiates markedly lower rates at any target coverage.

Mechanism Validation & Logprob-Nativity

A critical ablation with Claude-3-Haiku (no logprob access) establishes ECLIPSE as logprob-native. When log probabilities are replaced by heuristic proxies, AUC drops to 0.59 with coefficient magnitudes collapsing by 90–96% and $\Delta L$ flipping sign, annihilating the framework’s discriminative mechanism.

Figure 4: Coefficient comparison for GPT-3.5-turbo (real logprobs) and Claude-3-Haiku (proxies), exposing collapse of meaningful signal.

Feature Visualization

Distributions of core features clearly separate hallucinated from clean samples: higher entropy, lower capacity, and reduced capacity lift in hallucinated outputs.

Figure 5: Feature stratification—higher entropy and lower evidence capacity in hallucinated vs clean samples.

A scatter plot of semantic entropy vs capacity lift reveals near-linear separation, affirming the joint entropy–capacity trade-off as the essential discriminant.

Figure 6: Scatter of $H$ vs $\Delta L$, with fitted decision boundary clearly segregating clean and hallucinated answers.

Comparative Analysis and Implications

Qualitative comparison situates ECLIPSE among recent hallucination detectors: comparable AUCs to Semantic Entropy Probes (white-box), SelfCheckGPT (black-box), but with the distinct advantage of interpretability and reliance solely on logprob-accessible interfaces. Most importantly, ECLIPSE’s logprob-nativity links detection efficacy directly to structured token-level uncertainty, as opposed to crude aggregated confidences.

Practical implications include robust risk estimation for retrieval-augmented QA in financial automation, operational abstention for selective prediction, and interpretative insight into model behavior (notably overconfidence). The framework suggests that simple output confidence should not be used as a safety signal; rather, confidence in the face of ignored evidence should heighten risk estimates.

Theoretically, the convex entropy–capacity objective opens avenues for the design of optimal entropy controllers in generative LLMs, providing well-posed control for minimizing fatal hallucination risk.

Limitations and Future Directions

Evaluation is limited to a synthetic, single-domain dataset and requires real logprob access (not universal across APIs). Entropy estimation via low-sample clustering may underrepresent multimodal uncertainty. The framework could be confounded by adversarially coordinated evidence or distributions of naturally occurring hallucinations. Broader generalization, adversarial robustness, and explicit entropy control remain targets for subsequent research.

Conclusion

ECLIPSE demonstrates that hallucination detection in financial LLM QA can be rigorously framed and empirically validated as a problem of balancing output entropy against evidence capacity. The mechanism depends strictly on access to reliable, calibrated log probabilities—without which the discriminative signal collapses. The framework's interpretability, operational accuracy (92% reduction in hallucination rate at 30% coverage), and theoretical structure position it as a substantive contribution to calibration and reliability in high-stakes language modeling. Practical deployment requires API support for logprobs, robust entropy estimation, and broader validation across domains and naturally-occurring hallucinations for full operational assurance.