--- title: LLM Hallucinations url: https://www.emergentmind.com/topics/hallucination-in-large-language-models type: topic --- # LLM Hallucinations Large language models (LLMs) exhibit striking linguistic competence but are fundamentally prone to producing outputs that are either inaccurate or fabricated—a phenomenon now termed hallucination. This covers generative text that is fluent and syntactically correct but factually unsupported, incorrect, or unverifiable. Hallucination is increasingly recognized as a structural feature of deep learning models—particularly under the open-world assumption, where models confront an unbounded, ever-evolving environment and must generalize far beyond finite training data. Understanding the origins, theoretical bounds, typologies, detection/mitigation strategies, and design implications of hallucination in LLMs is central to both practical system reliability and the broader pursuit of artificial general intelligence. ## 1. Foundational Definitions and Theoretical Bounds In formal terms, let $X$ be the (possibly infinite) input space of token sequences, $Y$ the output space, $f:X\to Y$ the ground-truth world function, and $\hat{f}:X\to Y$ the mapping realized by an LLM post-training. Hallucination on input $x\in X$ occurs iff $\hat{f}(x)\neq f(x)$ [2510.05116]. The expected hallucination rate (or generalization error) under distribution $D$ is $$ L_D(\hat{f}) = \mathbb{P}_{(x,y)\sim D}\left[\hat{f}(x)\neq y\right]. $$ **Closed World Assumption:** $D_\mathrm{train} = D_\mathrm{test}$. In this regime, classical learning theory ensures that with enough data, hallucinations can be made arbitrarily rare. **Open World Assumption:** $D_\mathrm{test}$ is not constrained to equal $D_\mathrm{train}$; new inputs and tasks are encountered after training. In this regime, the No Free Lunch theorem dictates that there exist functions $f$ for which $\hat{f}(x)\neq f(x)$ for infinitely many $x$, and thus hallucinations are inevitable [2510.05116]. No finite training set can immunize an LLM against hallucinations in an unbounded environment. ## 2. Hallucination Typologies: Structural, Empirical, and Categorical ### 2.1 Formal and Behavioral Taxonomies - **Type I Hallucination (False Memorization):** Occurs when $x\in S_\mathrm{train}$ but $\hat{f}(x)\neq f(x)$. These are corrigible in principle by correcting the training set or fine-tuning [2510.05116]. - **Type II Hallucination (False Generalization):** $x\notin S_\mathrm{train}$ and $\hat{f}(x)\neq f(x)$. These reflect inevitable misgeneralization and cannot be eliminated under open-world conditions. Parallel taxonomies refine these distinctions: - **Intrinsic**: Output contradicts facts in the prompt or provided context (faithfulness failure). - **Extrinsic**: Output introduces unsupported or fabricated information not entailed in the context [2510.06265], often aligned with fabrication, imitative falsehood, or context drift [2310.03368]. Fine-grained categories have also been articulated: acronym ambiguity, numeric nuisance, generated golem (fabricated entities), virtual voice (misattributed quotes), geographic erratum, and temporal displacement [2310.04988]. ### 2.2 High-Confidence Failures (Delusions) A critical distinction is labeled as “delusion”: hallucinations generated with abnormal model confidence. Given a belief score $b(\hat{y})\in[0,1]$, a hallucinated response $\hat{y}$ is a delusion if $b(\hat{y}) > \tau$, where $\tau$ is the average belief over correct answers. Delusions are insensitive to supervision and self-reflection, and dominate error rates as model confidence calibration degrades [2503.06709]. ### 2.3 Cross-Linguistic and Task Variants Analysis in multilingual settings reveals that hallucination rates vary dramatically across languages, correlating with data/resource availability. Fact-conflicting hallucinations, especially in low-resource languages, are more prevalent and challenging to detect [2410.18270]. In fields like vision-language modeling, hallucination spans additional axes: object, attribute, and relationship errors [2404.14233]. ## 3. Root Causes: Model, Data, and Contextual Factors Hallucinations emerge throughout the LLM development pipeline [2510.06265]: - **Model-centric factors:** Transformer autoregressive decoding, self-attention’s lack of existential/temporal grounding [2509.16297], softmax bottlenecks, shortcut correlations in high-dimensional token spaces. - **Data-centric factors:** Web-corpus biases, knowledge conflicts, out-of-date information, and long-tail sparsity results in overgeneralization or imitative falsehoods [2510.06265, 2310.03368]. - **Contextual/inference factors:** Prompt ambiguity, domain/task distribution shift, retrieval pipeline inconsistencies in RAG setups, and exposure bias in autoregressive decoding [2510.06265, 2601.09929]. In NLI and QA settings, sentence-level memorization and learned corpus-level statistical patterns lead to both false entailment and factual hallucination even in deterministic setups [2305.14552]. ## 4. Detection Methodologies: Metrics and Pipelines Detection strategies span several methodologies [2510.06265, 2512.22416, 2502.15844]: - **Retrieval-based:** Compare LLM outputs to trusted knowledge bases or retrieved textual evidence. High-precision but computation-heavy, reliant on retriever quality. - **Uncertainty-based:** Token-level or semantic entropy signals, often calibrated with reference sets or model ensembles. These can be computed even in closed-box settings [2512.22416, 2601.09929]. - **Consistency/self-verification:** Re-asking questions with paraphrased prompts or mutated templates (metamorphic relations). Violations of internal consistency (e.g., contradiction under synonymic or antonymic reformulation) flag hallucinations [2502.15844]. - **Learning-based classifiers:** Supervised or semi-supervised discriminators trained to distinguish reliable vs. hallucinated outputs using embeddings, attention, and composite metrics [2407.04121, 2512.22416]. - **Multiple-testing hypothesis frameworks:** Apply FDR-controlled conformal testing using multiple independent uncertainty or similarity scores, providing provable guarantees on false-positive rates [2508.18473]. Segment-based (local) evaluation is critical in summarization and long-form generation, as hallucinations can concentrate in small spans while most of the text remains factual [2512.22416, 2404.14233]. ## 5. Mitigation and Management Strategies No mitigation strategy can eliminate hallucinations in the open world, but multiple approaches substantially reduce their prevalence or impact: - **Retrieval-augmented generation (RAG):** Fact grounding via external document integration at inference time reduces both ordinary hallucinations and delusions [2503.06709, 2510.06265]. - **Prompt engineering:** Instruction layering, in-context learning, chain-of-thought prompting, self-consistency voting, and explicit abstention instruction (refusal training) [2510.06265, 2310.06271]. - **Direct preference optimization (DPO), RLHF, and contrastive fine-tuning:** Preference datasets and structured loss objectives penalize unsupported generation, often with severity weighting [2404.14233]. - **Model calibration:** Temperature scaling, isotonic regression, and Bayesian post-hoc calibration to align model confidence with factual accuracy [2601.09929]. - **External consensus and multi-agent pipelines:** Multi-model debate/voting architectures significantly reduce the incidence of persistent, high-confidence hallucinations [2503.06709]. - **Entropy- and evidence-guided regeneration:** Automatic identification and rewriting of high-entropy or ungrounded spans using lower-vulnerability models or fact-checkers [2310.04988]. ## 6. Structural and Architectural Implications Recent theoretical and empirical arguments converge on the view that hallucination is structurally inevitable in existing transformer-based LLMs [2510.05116, 2509.16297]. Because self-attention fields lack existential or temporal grounding, models default to generating fluent but unmoored continuations whenever world information is absent or ambiguous. Proposals for architectural innovation include: - **Truth-constrained generation:** Embedding explicit verification or abstention drives alongside traditional autoregressive continuation, permitting calibrated refusal when outputs cannot be grounded [2509.16297]. - **Embedding symbolic and causal constraints:** Incorporating event order, causal relationship graphs, or affordance maps to shape possible continuations [2509.16297]. - **Hybrid parametric/non-parametric stacks:** Combining parametric neural models with symbolic retrieval or logic-based modules for stronger factual alignment [2510.06265]. Engineering for controllability, introspectability, and reliable uncertainty estimation is essential. Training separate hallucination detectors simply transfers the generalization problem unless detectors themselves are equipped with uncertainty awareness calibrated to the model’s knowledge boundaries [2510.05116]. ## 7. Evaluation Benchmarks and Open Challenges A diverse set of task-specific benchmarks now exist for measuring hallucination, with labeled datasets in QA (TruthfulQA, HaluEval, HalluQA), summarization (CNN/DailyMail, XSum, FactCC), dialogue (DialFact, WoW), and multilingual freeform generation (HalOmi, Mu-SHROOM) [2510.06265, 2310.03368, 2410.18270, 2502.13622]. Key challenges include: - **Universal detection and robust cross-domain generalization** for detectors, particularly in low-resource and non-English contexts [2410.18270, 2512.22416]. - **Granularity in hallucination localization**, including segment/sentence/claim-level detection [2404.14233, 2502.13622]. - **Explainability and transparency** about the provenance and uncertainty of output claims [2510.06265, 2601.09929]. - **Dynamic lifelong learning and continual calibration** as LLM deployments encounter new domains and information [2510.05116]. Evaluation increasingly incorporates both automatic metrics (AUROC, F1, calibration error) and LLM/human-as-judge protocols for adjudicating factuality at scale. There is convergent emphasis on systemic, root cause–aware pipelines rather than one-off patches or ad hoc filters [2601.09929]. --- **References** - [2510.05116] Hallucination is Inevitable for LLMs with the Open World Assumption - [2512.22416] Hallucination Detection and Evaluation of Large Language Model - [2310.01469] LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examples - [2503.06709] Delusions of Large Language Models - [2510.06265] Large Language Models Hallucination: A Comprehensive Survey - [2404.14233] Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedback - [2509.16297] How Large Language Models are Designed to Hallucinate - [2407.04121] Hallucination Detection: Robustly Discerning Reliable Answers in Large Language Models - [2310.06271] Towards Mitigating Hallucination in Large Language Models via Self-Reflection - [2305.14552] Sources of Hallucination by Large Language Models on Inference Tasks - [2403.20009] On Large Language Models' Hallucination with Regard to Known Facts - [2310.03368] Evaluating Hallucinations in Chinese Large Language Models - [2508.18473] Principled Detection of Hallucinations in Large Language Models via Multiple Testing - [2502.13622] REFIND at SemEval-2025 Task 3: Retrieval-Augmented Factuality Hallucination Detection in Large Language Models - [2601.09929] Hallucination Detection and Mitigation in Large Language Models - [2410.18270] Multilingual Hallucination Gaps in Large Language Models - [2310.04988] The Troubling Emergence of Hallucination in Large Language Models -- An Extensive Definition, Quantification, and Prescriptive Remediations - [2502.15844] Hallucination Detection in Large Language Models with Metamorphic Relations Source: https://www.emergentmind.com/topics/hallucination-in-large-language-models