---
title: Thinking-Induced Hallucination
url: https://www.emergentmind.com/topics/thinking-induced-hallucination
type: topic
---

# Thinking-Induced Hallucination

Thinking-induced hallucination denotes hallucination that is produced or amplified by a model’s own deliberative process—chain-of-thought, self-reflection, test-time reasoning, or prompt-induced conceptual elaboration—rather than by a final answer in isolation. In factuality-oriented question answering, it names the case where explicit thinking overturns a correct non-thinking answer and introduces factual drift [2607.05861]. In large vision-language and multimodal reasoning models, it denotes visually ungrounded content arising from flawed or unsupported internal reasoning during step-by-step generation [2505.07172]. More broadly, recent work also uses the term to discuss a spectrum in which departures from fact may be either defective or, under controlled conditions, creatively valuable [2512.21635].

## 1. Terminology and conceptual scope

Recent work does not use the term in a fully uniform way. Some papers restrict it to accuracy regressions induced by explicit reasoning, while others extend it to multimodal grounding failures, agentic planning errors, or even creativity-oriented departures from strict factuality.

| Setting | Characteristic use of the term | Representative source |
|---|---|---|
| Factual QA with LRMs | Explicit thinking changes a correct direct answer into an incorrect, unsupported prediction | [2607.05861] |
| LVLM/MLLM reasoning | Hallucination arises due to flawed or unsupported internal reasoning during CoT | [2505.07172] |
| Embodied planning | Reasoning fills in objects or steps that are absent from the scene | [2506.15065] |
| Tool-using agents | Step-by-step reasoning increases fabricated or inappropriate tool use | [2510.22977] |
| Creativity evaluation | Hallucinations are split into Intelligent Hallucinations and Defective Hallucinations | [2512.21635] |

A major reason for this terminological spread is that hallucination itself remains difficult to define. A logic-based synthesis of prior work classifies output quality through entailment, contradiction, logical independence, and omission, distinguishing cases such as intrinsic hallucination, extrinsic hallucination, and omission. That framework also argues that current classifications are limited in open-ended settings and may underdetect failures rooted in reasoning, implicature, presupposition, or withholding information; it explicitly points to belief-, desire-, and intention-based formalisms as a future direction for capturing deeper “thinking” failures [2401.07897].

Within this broader landscape, HIC-Bench formalizes a particularly expansive interpretation. It distinguishes **Intelligent Hallucinations (IH)**, defined as responses with **Originality $\geq 4$**, **Value $\geq 4$**, and **Feasibility $\geq 3$** despite partial deviation from fact, from **Defective Hallucinations (DH)**, which exhibit factual errors, logical inconsistencies, violation of scientific principles, or lack of value. Its associated **Intelligent-Fidelity Score (IFS)** makes explicit that some work treats thinking-induced hallucination not only as failure, but also as a controllable trade-off between creativity and correctness [2512.21635].

## 2. Failure mechanisms in reasoning models

A central mechanistic account treats explicit thinking as a perturbation of the model’s direct-answer tendency. MARGO formulates thinking in factual QA as a **thinking residual** over the model’s non-thinking response: the residual can recover missing knowledge, but it can also introduce unsupported associations. The paper operationalizes thinking-induced hallucination by comparing thinking and non-thinking modes and identifying the transition from correct non-thinking to incorrect thinking; it reports a significant proportion of such cases, including approximately **7–8% for Qwen3-8B**, and qualitatively notes traces containing hedges such as “But wait” and “Hmm, not sure,” indicating instability triggered by reasoning [2607.05861].

A complementary line of work studies long-chain reasoning as a source of meta-cognitive error amplification. Auditing Meta-Cognitive Hallucinations in Reasoning Large Language Models distinguishes **Type I (Seen but Unlearned)** from **Type II (Unseen or Incorrect)** hallucinations and argues that reflective reasoning can iteratively reinforce bias and error rather than correct it. In long-CoT settings, hallucinated chains are longer, reflection frequency is about **2.12x higher**, hedging and hesitation increase by more than **200%**, and direct interventions often fail to repair the chain: despite edit acceptance rates above **70%**, only about **22.5%** of edits fully reverse the hallucination. This resistance to correction is termed **chain disloyalty** [2505.13143].

Prompt structure can itself induce a reasoning mode that destabilizes conceptual grounding. The HIP/HQP framework shows that prompts fusing semantically distant concepts, such as the periodic table and tarot divination, systematically trigger less coherent and more hallucinated outputs than matched controls. These prompt-induced hallucinations are presented not as random noise but as responses to semantic “instability zones.” Model behavior is heterogeneous: **DeepSeek-R1** and **DeepSeek** receive the highest hallucination scores under fusion prompts, whereas **Gemini2.5Pro** often refuses the fusion and yields much lower scores [2505.00557].

A more general theoretical account situates both human and machine hallucination in predictive inference. On that view, humans hallucinate when strong priors or ambiguous input lead the system to settle on expectation rather than evidence, while LLMs hallucinate because auto-regressive generation fills informational gaps using learned statistical regularities rather than truth verification. The shared mechanism is gap-filling under uncertainty, though the substrate and correction pathways differ [2503.05806].

## 3. Multimodal, agentic, and embodied manifestations

In multimodal reasoning models, longer reasoning chains often weaken perceptual grounding rather than strengthen it. “More Thinking, Less Seeing?” reports that as generations become longer, models allocate less attention to visual tokens and more to instruction and language tokens, leading to drift away from image-grounded content. The paper introduces **RH-AUC** to quantify the balance between reasoning performance and perceptual fidelity across reasoning lengths, and finds a non-monotonic trade-off: moderate-length reasoning can be optimal, whereas excessive reasoning degrades both grounding and accuracy [2505.21523].

This multimodal phenomenon is closely related to the distinction between **thinking while looking** and **thinking before looking**. “Thinking Before Looking” argues that standard multimodal CoT tightly entangles image and text during reasoning, encouraging stereotypical reasoning patterns and visual-prior bias. Its diagnosis is that simultaneous access to the image during reasoning can aggravate reasoning-related hallucination instead of mitigating it [2411.12591].

In tool-using agents, the same pattern appears in a different form. SimpleToolHalluBench isolates two failure modes—**No Tool Available (NTA)** and **Distractor Tool (DT)**—and shows that progressively enhancing reasoning through RL increases tool hallucination proportionally with task performance gains. The effect is method-agnostic: it appears after RL, after supervised fine-tuning or distillation, and even when reasoning is elicited only at inference. For **Qwen3-8B**, simply enabling thinking changes **$R_{\text{NTA}}$** from **4.1%** to **5.4%** and **$R_{\text{DT}}$** from **36.2%** to **56.8%** [2510.22977].

Embodied agents exhibit a grounding-specific variant. HEAL defines hallucination as generation of plans, object references, or state references not grounded in the observed physical environment, and constructs a probing set that can induce hallucination rates up to **40x higher** than base prompts. The most severe failures occur under **Scene Task Contradiction**, where the instruction requires absent objects and the planner reasons as if they exist. The paper emphasizes that these are not merely copying errors: reasoning traces explicitly rationalize why a missing object “should” exist and proceed accordingly [2506.15065].

Video understanding adds temporal complexity. HAVEN organizes hallucination in video LMMs along **hallucination causes**, **hallucination aspects**, and **question formats**, yielding about **6K questions**. It shows that CoT often improves accuracy and usually reduces bias, but the relationship is not uniform across models; for **LLaMA-VID-7B**, CoT improves accuracy while increasing bias [2503.19622]. Adversarial image generation reveals an even more direct attack on the reasoning process: GHOST generates natural-looking, object-free images that reroute MLLM reasoning, achieving a hallucination success rate of about **32.4%** on **GLM-4.1V-Thinking** while keeping the target object absent [2509.25178].

## 4. Evaluation and diagnosis beyond final-answer accuracy

Because thinking-induced hallucination often resides in the reasoning trace rather than the final answer, recent evaluation frameworks explicitly score intermediate cognition, grounding drift, or abstention failures.

| Framework | Setting | Main diagnostic focus |
|---|---|---|
| **RACE** [2506.04832] | LRMs | Inter-sample reasoning consistency, answer uncertainty, reasoning-answer alignment, internal coherence |
| **MM-THEBench** [2601.22735] | Reasoning MLLMs | Step-level hallucinations in perception, knowledge, and reasoning; intermediate CoT correctness lower than final answer accuracy |
| **RH-Bench / RH-AUC** [2505.21523] | Multimodal reasoning length | Change in perceptual accuracy as reasoning length varies |
| **SimpleToolHalluBench** [2510.22977] | Tool-using agents | Hallucination when no correct tool exists or only distractor tools are available |
| **HAVEN** [2503.19622] | Video understanding | Hallucination causes, aspects, and question formats; accuracy and bias score |
| **HEAL probing set** [2506.15065] | Embodied planning | Scene-task inconsistency, measured with CHAIR and POPE |
| **HIP/HQP** [2505.00557] | Prompt-induced hallucination | Hallucination score from 0 to 10 under semantically unstable conceptual fusion |

These frameworks converge on a common result: answer-level correctness is not a sufficient proxy for faithful reasoning. RACE explicitly targets cases where sampled answers look consistent while their chains of thought are contradictory, unsupported, or generic, and reports that **inter-reasoning consistency** contributes the largest single-signal improvement over answer-only baselines [2506.04832]. MM-THEBench likewise reports that **intermediate CoT correctness is significantly lower than final answer accuracy**, and that **reasoning hallucinations** are more strongly correlated with incorrect final answers than pure perception errors [2601.22735].

A second diagnostic theme is abstention. In HEAL, the correct behavior under **Task Relevant Object Removal** and **Scene Task Contradiction** is not creative substitution but refusal, rejection, or an empty plan [2506.15065]. SimpleToolHalluBench makes the same point in agentic form: trustworthy reasoning includes knowing when no legitimate tool action exists [2510.22977].

## 5. Mitigation strategies and their trade-offs

One major strategy is to inject grounded structure before or during reasoning. **Re-Critic** augments multimodal instruction tuning with a **Visual Rationale Synthesizer (VCIT)** and an **in-context self-critic mechanism** for preference tuning. The rationale is inserted into the input prompt rather than emitted as output, and the self-critic selects better response pairs for **DPO**. On hallucination benchmarks, the method improves average performance by **+6.2% for LLaVA-v1.5 (7B)** and **+2.7% for InternVL 2.0 (2B)**; with **LLaVA-80K**, it yields a **31.8% improvement** over baseline. On **Object HalBench** for **LLaVA-v1.5**, response-level hallucination decreases from **56.4%** to **49.6%** and mention-level hallucination from **27.9%** to **24.9%**. The same model family also improves on **MathVista**, indicating that the gains extend beyond hallucination-specific tasks [2505.07172].

A second line uses counterfactual or contrastive prompting at inference time. **Counterfactual Inception** generates plausible but absent keywords, filters them by **CLIP** similarity and **NLI contradiction score**, and re-prompts the model so that it reasons against carefully chosen alternatives. Across models, this reduces hallucination without retraining; in human consistency scoring, average hallucinatory keywords per image decrease from **1.7** to **0.81**, while consistency increases from **3.3** to **4.13** [2403.13513]. **VIC** follows a different route: it builds a reasoning chain from textual context alone before introducing the image. On **MMVP**, **Gemini 1.5 Pro** improves from **0.420** to **0.553**, and on **HallusionBench**, **GPT-4o** improves from **0.626** to **0.692** [2411.12591].

A third family regulates when thinking should occur at all. **MARGO** trains with mixed-mode rollout groups containing both thinking and non-thinking trajectories, rewarding reasoning only when it adds factual value beyond direct answering. It raises average accuracy from **22.81%** to **25.49%** on **Qwen3-4B** and from **27.59%** to **29.57%** on **Qwen3-8B**, while preserving mathematical reasoning ability on **AMC23**, **AIME24**, and **AIME25** [2607.05861]. **HaluSearch** frames generation as tree search with a self-evaluation reward model and a fast/slow thinking switch; on **HaluEval-QA** with **Llama3.1-8B-Instruct**, direct inference attains **35.6%**, whereas **MCTS (HaluSearch)** reaches **45.4%** [2501.01306].

Mitigation remains domain-dependent and often costly. In video LMMs, **SRFT** plus **TDPO** improves hallucination accuracy by **7.65%** over the baseline and reduces the bias score by **4.5%** on HAVEN [2503.19622]. In embodied planning, **Self-Correcting Woodpecker (SCW)** outperforms **Knowledge-Augmented Feedback (KAF)**, but hallucination remains high in the hardest **SceneTaskCon** setting [2506.15065]. For tool hallucination, **DPO** can reduce **$R_{\text{NTA}}$** from **90.2%** to **55.8%** and **$R_{\text{DT}}$** from **100.0%** to **71.4%**, yet **SynTool Reward** drops from **0.45** to **0.34**, making the reliability-capability trade-off explicit [2510.22977]. GHOST shows that adversarially generated negative examples can also be used constructively: fine-tuning **Qwen2.5-VL** on GHOST images reduces hallucination on unseen **GHOST-LLaVA** images from **52.6%** to **7.0%**, on **GHOST-GLM** from **63.1%** to **7.3%**, while **POPE** rises from **88.5** to **90.7** [2509.25178].

## 6. Human processing, theory, and contested implications

The study of thinking-induced hallucination is not limited to machine behavior. A neuroimaging study using EEG and ERP analysis records signals from **27 participants** performing image-description verification on MLLM-generated captions. It finds that correctly recognized hallucinations elicit enhanced **N100**, **P200**, **N400**, and **P600** responses, whereas hallucinations that are **misjudged as correct** are neurally indistinguishable from accurate descriptions: **HalluWrong vs. NoHallu** shows no statistically significant differences in any ERP component or region. EEG-based prediction achieves **AUC > 0.9 within subject** only when participants recognize the hallucination; when they are misled, performance falls to chance. The paper interprets this as failure to trigger the standard neurocognitive fact-verification pathway [2605.16953].

At a broader theoretical level, predictive-processing accounts treat hallucination as a consequence of advanced inference under uncertainty. One recent comparison argues that both humans and LLMs are predictive systems that interpolate or extrapolate missing information; in humans this occurs through hierarchical prediction and prediction-error correction, while in LLMs it occurs through auto-regressive next-token modeling without intrinsic truth checking [2503.05806]. A very different and explicitly speculative line models hallucination in schizophrenia as desynchronized time evolution of percept basis states in an infinite-dimensional **Hilbertian consciousness**, proposing that thought-driven activation of perceptual basis vectors can generate hallucinatory experience when internal evolution is out of sync with real-world time [1706.03619].

The normative status of hallucination is also contested. HIC-Bench argues that reducing hallucination indiscriminately may suppress creative potential, and reports a nonlinear relationship between **IH** and **DH** rather than a simple antagonism. Under **DHP + RCP**, **GPT-4o** reaches an **IH ratio of 21.1%** with **DH down to 0.3%**, and the automated IH/DH labeling is supported by human verification with precision and recall above **85%** for both classes [2512.21635]. This does not negate the harms of defective hallucination, but it complicates any definition that equates all evidence-exceeding generation with defect.

Taken together, these studies undermine three recurring assumptions. More reasoning does not reliably produce more truth; correct final answers do not guarantee faithful intermediate reasoning; and hallucination is not always a single-category pathology. Current work therefore treats thinking-induced hallucination as an interaction between reasoning depth, grounding, metacognition, prompting, and task structure. The unresolved problem is how to preserve the benefits of explicit thinking while preventing the reasoning process itself from becoming an independent source of error.

Source: https://www.emergentmind.com/topics/thinking-induced-hallucination