Reasoning-Driven Hallucination in AI Models

• Reasoning-driven hallucination is the phenomenon where AI models generate logically coherent but factually unsupported chains-of-thought due to inference errors in multi-step reasoning.
• It spans various domains including language, vision–language, and video understanding, manifesting through spatial, logical, and factual inconsistencies.
• Mitigation strategies such as PLR+FAE, RHD, and curriculum-based reward shaping aim to detect and reduce these hallucinations in AI systems.

Reasoning-Driven Hallucination

Reasoning-driven hallucination denotes the phenomenon in which LLMs, multimodal LLMs (MLLMs), or specialized reasoning systems generate responses that are logically coherent but factually unsupported due to errors or biases in multi-step inference, chain-of-thought generation, or decision-making sub-processes. Unlike token-level hallucination, where isolated outputs are factually incorrect, reasoning-driven hallucination is embedded in the model’s structured reasoning traces, leading to persuasive but faulty conclusions. This failure mode has been demonstrated across language, vision–language, and video understanding domains, and is now a primary reliability bottleneck for foundation models (Pu et al., 23 Nov 2025, Hu et al., 15 Sep 2025, Li et al., 8 Oct 2024, Sun et al., 19 May 2025).

1. Formal Definitions and Mechanisms

Reasoning-driven hallucination encompasses logically structured but factually invalid reasoning, originating not from the omission of evidence, but from flaws or shortcuts in the inference process. The key defining features are:

• The response comprises a chain-of-thought or multi-step description that is internally consistent yet ungrounded in the input evidence or world knowledge.
• Hallucination may occur at any step: fabricated intermediate claims, factual inconsistencies, context-step mismatches, or entirely invented logical sub-chains.
• The phenomenon is distinct from perception-induced hallucination, where the initial evidence extraction is erroneous. In reasoning-driven hallucination, perceptual grounding is correct, but the logical integration falters (Dong et al., 30 May 2025).

Formally, for a chain $\mathcal{C} = (c_1, \ldots, c_K)$, reasoning-driven hallucination is present when $\exists\, k$ such that $c_k$ is not entailed by input evidence and prior steps, yet is included in the chain (Sun et al., 19 May 2025, Dong et al., 30 May 2025). In video understanding, this arises when a model generates a description $d_t$ (with timestamped video support) that is not verifiable or grounded in the actual observed segment, yet is treated as valid evidence for final answer generation (Pu et al., 23 Nov 2025).

2. Taxonomies and Categories

Recent works codify subtypes of reasoning-driven hallucinations across domains:

• Mathematical/Logical Reasoning (FG-PRM): Fabrication, Factual Inconsistency, Context Inconsistency, Instruction Inconsistency, Logical Inconsistency, Logical Error (Li et al., 8 Oct 2024).
• Vision–LLMs (MIRAGE): Spatial Hallucination, Logical Hallucination, Factuality Hallucination, Fabrication Hallucination (Dong et al., 30 May 2025). Here, models fail at geometric relations, logical consistency, world knowledge, or outright invent entities.
• Video Reasoning (Video-PLR): Attribute Modification, Quantity Modification, Action Substitution, Detail Conflation, Temporal Reordering (Pu et al., 23 Nov 2025).

Table 1: Representative Subtypes of Reasoning-Driven Hallucination

Domain	Subtypes
Mathematical Reasoning	Fabrication, Factual/Context/Instruction/Logical (In)consistency, Logical Error
Vision–Language	Spatial, Logical, Factuality, Fabrication Hallucination
Video Understanding	Attribute & Quantity Modification, Action Substitution, Temporal Reordering

Errors can be “intrinsic” (inconsistent with model’s internal state or instructions) or “extrinsic” (inconsistent with world or physical evidence) (Li et al., 8 Oct 2024, Dong et al., 30 May 2025).

3. Diagnosis and Detection Methodologies

A core challenge is detecting hallucinations that are deeply embedded in plausible reasoning traces. Several methodologies have emerged:

• Perception-Loop Reasoning (PLR) + Factual-Aware Evaluator (FAE): Instead of describing the video in a single step, PLR alternates between stepwise, timestamped evidence extraction and local segment analysis. Each evidence segment $e_t$ is immediately scored by a factuality evaluator (FAE), producing anti-hallucination rewards for reinforcement learning. This shuts down invented or unsupported subchains early in the reasoning process (Pu et al., 23 Nov 2025).
• Reasoning Subspace Projection (HARP): Decomposes hidden states into semantic and reasoning subspaces via SVD on the model’s unembedding matrix. Hallucinations are detected by projecting each token's hidden state onto the low-rank “reasoning” subspace (≈5% of model dimension), followed by a lightweight classifier (Hu et al., 15 Sep 2025).
• Fine-Grained Process Reward Models (FG-PRM): Six Process Reward Models, each targeting a specific hallucination category, are trained using synthetic data injected with precise hallucination instances, enabling step-level detection and solution ranking in mathematical reasoning (Li et al., 8 Oct 2024).
• Reasoning Hallucination Detection (RHD): Computes a Reasoning Score for each token or step, defined as the divergence between logits from late layers and earlier layers (“deep vs. shallow reasoning”). Early-stage fluctuations and incorrect late-step backtracking to flawed inference paths are predictive of hallucination. The RHD framework integrates this with attention and perplexity correlations for robust detection (Sun et al., 19 May 2025).
• IRIS (Unsupervised): Extracts contextualized representations of model-internal reasoning traces after self-prompted verification, then classifies them using a three-layer MLP probe trained only on model-internal uncertainty (Srey et al., 12 Sep 2025).

4. Empirical Analysis and Failure Modes

Systematic analyses and benchmarks such as MIRAGE, RH-Bench, and HaloQuest reveal characteristic patterns:

• Model and Data Scale: Hallucination rates decrease with model scale and data scale, except for spatial hallucinations which remain resistant (e.g., logical hallucination: 3B at 78.9% vs. 72B at 47.7%) (Dong et al., 30 May 2025).
• Training Regime Effects: RL-only pipelines amplify hallucinations (especially after long-chain-of-thought fine-tuning), while full supervised + verifiable-reward RL mitigates them (Li et al., 30 May 2025, Yao et al., 29 May 2025).
• Behavioral Signatures: Flaw repetition (looping over similar but incorrect inference steps) and think-answer mismatch (final answer diverges from the body of reasoning) increase under shallow or “imitation-only” training (Yao et al., 29 May 2025).
• Attention and Reasoning Drift: As reasoning chains are extended, models’ attention to perceptual tokens decays, and reliance on language/concept priors grows, leading to so-called “reasoning drift”—a core mechanism for hallucination (Liu et al., 23 May 2025, Lu et al., 11 Oct 2025).
• Meta-Cognitive Hallucination and Chain Disloyalty: LLMs reinforce false claims through self-reflection and hedging, causing error propagation (“chain disloyalty”) even when corrections are introduced early (Lu et al., 19 May 2025).

For tool-augmented LLMs, stronger reasoning induces increased “tool hallucination”—spurious tool calls in the absence of an appropriate tool, aggravated by RL or chain-of-thought prompting (Yin et al., 27 Oct 2025).

5. Quantitative Metrics and Benchmarking

Several metrics and diagnostics quantify reasoning-driven hallucination:

• Task-level accuracy and hallucination rate: Standard metric—1 minus accuracy, as human/LLM-judged correct answer rate (Yao et al., 29 May 2025, Dong et al., 30 May 2025).
• Intermediate Chain Factuality: Precision, recall, and F1 for step- or claim-level agreement between model chains and verified chains (Dong et al., 30 May 2025).
• RH-AUC: Area under the Reasoning–Hallucination curve, summarizing the trade-off between reasoning depth and perceptual fidelity as chain length varies (Liu et al., 23 May 2025).
• Reasoning Score (RHD): Average stepwise JSD divergence, coefficient of variation, backtracking score, and correlation with perplexity—combined into a composite score for binary detection and multi-trace ranking (Sun et al., 19 May 2025).
• Span-level F1: In hallucination span detection, F1 between predicted and reference hallucinated text (Su et al., 2 Oct 2025).

Table 2: Illustrative Quantitative Results

Setting	Hallucination Rate	Intervention	Factuality Gain
Video-PLR-7B	+1.7 pp over SOTA	PLR+FAE (timestamp+reward)	FAE: +7–9%
MIRAGE-7B	-15.4 pp (logical)	CRFT + CHI (curricular RL)	Logos: +8–10 pp
RHD–Math domain	AUC 0.798	RHD vs. SelfCheckGPT	+9% MC3 ranking

6. Mitigation Strategies

Mitigating reasoning-driven hallucination requires structured intervention at reasoning step granularity:

• Looped Evidence and Per-Step Fact-Checking: PLR+FAE enforces that every inference step is grounded in timestamped evidence, penalizing fabricated or redundant content immediately (Pu et al., 23 Nov 2025).
• Reward Shaping in RL: Potential-based shaping on reasoning-depth signals (e.g., RHD’s Reasoning Score) regularizes CoT generation, balancing deep inference with factual accuracy and avoiding spurious optima (Sun et al., 19 May 2025, Li et al., 30 May 2025).
• Fine-Grained Process Rewards: FG-PRM uses hallucination-type specialists, improving solution selection and out-of-distribution generalization via synthetic data augmentation (Li et al., 8 Oct 2024).
• Attention Control: Functional rescaling of perception and reasoning attention heads at inference time reduces reasoning drift and perceptual bias in vision–LLMs, with <1% runtime overhead (Lu et al., 11 Oct 2025).
• Curriculum RL and Collaborative Hints: Multi-stage training plus topic- and question-specific hint prompting refines logical consistency in chains, particularly in curriculum-constructed benchmarks (Dong et al., 30 May 2025).
• Self-Consistency Filtering: SSC rejects inconsistent or hallucinated intermediary reasoning paths by enforcing multi-sample agreement at step level (Liu et al., 13 Apr 2025).
• Multi-Agent Orthogonalization: Modular rationality pipelines—separate retrieval, tool orchestration, and validation agents—yield robust reduction in logical composition errors in high-stakes domains (e.g., ophthalmology) (Pan et al., 24 Jul 2025).
• Explicit Abstention and Defer Mechanisms: Absence-aware model outputs (e.g., “I cannot answer”) are advocated as architectural and training objectives for future truth-constrained systems (Ackermann et al., 19 Sep 2025, Yin et al., 27 Oct 2025).

7. Limitations and Ongoing Research

Empirical and theoretical analyses highlight several frontiers:

• Residual Reasoning Hallucination as a Structural Limitation: Transformer autoregression with flat self-attention is inherently vulnerable to generating plausible, non-grounded inference chains; current mitigation only masks the underlying deficit (Ackermann et al., 19 Sep 2025).
• Mode-Specific Tradeoffs: Reasoning-augmented (CoT) detection improves average accuracy but reduces low-FPR recall; best overall reliability may require hybrid or ensemble approaches (Chegini et al., 23 Oct 2025).
• Detection–Mitigation Feedback: Many advanced detectors require high-quality step-wise supervision or synthetic data, limiting transfer to new domains or out-of-distribution settings (Sun et al., 19 May 2025).
• Persistent Error Modalities: Spatial/hard perceptual reasoning remains a bottleneck in both VLMs and MLLMs, even with scaling and advanced mitigation (Dong et al., 30 May 2025).
• Tool Use Failure: Current reasoning enhancement strategies (RL, distillation, structured prompting) often increase spurious tool hallucination unless capacity–reliability is directly regularized (Yin et al., 27 Oct 2025).
• Chain Disloyalty and Metacognitive Failure: Flawed reasoning is compounded by models’ tendency to reinforce initial errors through overconfident reflection, which is not mitigated by local correction or black-box audit (Lu et al., 19 May 2025).

Research is now focused on aligning internal reasoning dynamics with external truth constraints, automating fine-grained claim validation, and developing architectures capable of abstention and grounding-aware inference (Hu et al., 15 Sep 2025, Ackermann et al., 19 Sep 2025, Pu et al., 23 Nov 2025).

---

References:

• "Alternating Perception-Reasoning for Hallucination-Resistant Video Understanding" (Pu et al., 23 Nov 2025)
• "HARP: Hallucination Detection via Reasoning Subspace Projection" (Hu et al., 15 Sep 2025)
• "Fine-grained Hallucination Detection and Mitigation in LLM Mathematical Reasoning" (Li et al., 8 Oct 2024)
• "Detection and Mitigation of Hallucination in Large Reasoning Models: A Mechanistic Perspective" (Sun et al., 19 May 2025)
• "MIRAGE: Assessing Hallucination in Multimodal Reasoning Chains of MLLM" (Dong et al., 30 May 2025)
• "More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Models" (Liu et al., 23 May 2025)
• "Learning to Reason for Hallucination Span Detection" (Su et al., 2 Oct 2025)
• "Reasoning's Razor: Reasoning Improves Accuracy but Can Hurt Recall at Critical Operating Points in Safety and Hallucination Detection" (Chegini et al., 23 Oct 2025)
• "The Reasoning Trap: How Enhancing LLM Reasoning Amplifies Tool Hallucination" (Yin et al., 27 Oct 2025)
• "Auditing Meta-Cognitive Hallucinations in Reasoning LLMs" (Lu et al., 19 May 2025)
• "HaloQuest: A Visual Hallucination Dataset for Advancing Multimodal Reasoning" (Wang et al., 22 Jul 2024)
• "Enhancing Mathematical Reasoning in LLMs with Self-Consistency-Based Hallucination Detection" (Liu et al., 13 Apr 2025)
• "Are Reasoning Models More Prone to Hallucination?" (Yao et al., 29 May 2025)
• "Unsupervised Hallucination Detection by Inspecting Reasoning Processes" (Srey et al., 12 Sep 2025)
• "EH-Benchmark Ophthalmic Hallucination Benchmark and Agent-Driven Top-Down Traceable Reasoning Workflow" (Pan et al., 24 Jul 2025)
• "How LLMs are Designed to Hallucinate" (Ackermann et al., 19 Sep 2025)