Thinking-Induced Hallucination

• Thinking-induced hallucination is hallucination produced by a model’s internal chain-of-thought, leading to factual drift and sometimes creative outputs.
• Research highlights diverse failure modes, including flawed internal reasoning, multimodal grounding failures, and agentic planning errors, diagnosed via tailored evaluation frameworks.
• Mitigation strategies involve grounded prompting, counterfactual reasoning, and controlled reasoning modes to balance factuality and creativity in AI systems.

Thinking-induced hallucination denotes hallucination that is produced or amplified by a model’s own deliberative process—chain-of-thought, self-reflection, test-time reasoning, or prompt-induced conceptual elaboration—rather than by a final answer in isolation. In factuality-oriented question answering, it names the case where explicit thinking overturns a correct non-thinking answer and introduces factual drift (Wang et al., 7 Jul 2026). In large vision-language and multimodal reasoning models, it denotes visually ungrounded content arising from flawed or unsupported internal reasoning during step-by-step generation (Yang et al., 12 May 2025). More broadly, recent work also uses the term to discuss a spectrum in which departures from fact may be either defective or, under controlled conditions, creatively valuable (Yang et al., 25 Dec 2025).

1. Terminology and conceptual scope

Recent work does not use the term in a fully uniform way. Some papers restrict it to accuracy regressions induced by explicit reasoning, while others extend it to multimodal grounding failures, agentic planning errors, or even creativity-oriented departures from strict factuality.

Setting	Characteristic use of the term	Representative source
Factual QA with LRMs	Explicit thinking changes a correct direct answer into an incorrect, unsupported prediction	(Wang et al., 7 Jul 2026)
LVLM/MLLM reasoning	Hallucination arises due to flawed or unsupported internal reasoning during CoT	(Yang et al., 12 May 2025)
Embodied planning	Reasoning fills in objects or steps that are absent from the scene	(Chakraborty et al., 18 Jun 2025)
Tool-using agents	Step-by-step reasoning increases fabricated or inappropriate tool use	(Yin et al., 27 Oct 2025)
Creativity evaluation	Hallucinations are split into Intelligent Hallucinations and Defective Hallucinations	(Yang et al., 25 Dec 2025)

A major reason for this terminological spread is that hallucination itself remains difficult to define. A logic-based synthesis of prior work classifies output quality through entailment, contradiction, logical independence, and omission, distinguishing cases such as intrinsic hallucination, extrinsic hallucination, and omission. That framework also argues that current classifications are limited in open-ended settings and may underdetect failures rooted in reasoning, implicature, presupposition, or withholding information; it explicitly points to belief-, desire-, and intention-based formalisms as a future direction for capturing deeper “thinking” failures (Deemter, 2024).

Within this broader landscape, HIC-Bench formalizes a particularly expansive interpretation. It distinguishes Intelligent Hallucinations (IH), defined as responses with Originality $\geq 4$, Value $\geq 4$, and Feasibility $\geq 3$ despite partial deviation from fact, from Defective Hallucinations (DH), which exhibit factual errors, logical inconsistencies, violation of scientific principles, or lack of value. Its associated Intelligent-Fidelity Score (IFS) makes explicit that some work treats thinking-induced hallucination not only as failure, but also as a controllable trade-off between creativity and correctness (Yang et al., 25 Dec 2025).

2. Failure mechanisms in reasoning models

A central mechanistic account treats explicit thinking as a perturbation of the model’s direct-answer tendency. MARGO formulates thinking in factual QA as a thinking residual over the model’s non-thinking response: the residual can recover missing knowledge, but it can also introduce unsupported associations. The paper operationalizes thinking-induced hallucination by comparing thinking and non-thinking modes and identifying the transition from correct non-thinking to incorrect thinking; it reports a significant proportion of such cases, including approximately 7–8% for Qwen3-8B, and qualitatively notes traces containing hedges such as “But wait” and “Hmm, not sure,” indicating instability triggered by reasoning (Wang et al., 7 Jul 2026).

A complementary line of work studies long-chain reasoning as a source of meta-cognitive error amplification. Auditing Meta-Cognitive Hallucinations in Reasoning LLMs distinguishes Type I (Seen but Unlearned) from Type II (Unseen or Incorrect) hallucinations and argues that reflective reasoning can iteratively reinforce bias and error rather than correct it. In long-CoT settings, hallucinated chains are longer, reflection frequency is about 2.12x higher, hedging and hesitation increase by more than 200%, and direct interventions often fail to repair the chain: despite edit acceptance rates above 70%, only about 22.5% of edits fully reverse the hallucination. This resistance to correction is termed chain disloyalty (Lu et al., 19 May 2025).

Prompt structure can itself induce a reasoning mode that destabilizes conceptual grounding. The HIP/HQP framework shows that prompts fusing semantically distant concepts, such as the periodic table and tarot divination, systematically trigger less coherent and more hallucinated outputs than matched controls. These prompt-induced hallucinations are presented not as random noise but as responses to semantic “instability zones.” Model behavior is heterogeneous: DeepSeek-R1 and DeepSeek receive the highest hallucination scores under fusion prompts, whereas Gemini2.5Pro often refuses the fusion and yields much lower scores (Sato, 1 May 2025).

A more general theoretical account situates both human and machine hallucination in predictive inference. On that view, humans hallucinate when strong priors or ambiguous input lead the system to settle on expectation rather than evidence, while LLMs hallucinate because auto-regressive generation fills informational gaps using learned statistical regularities rather than truth verification. The shared mechanism is gap-filling under uncertainty, though the substrate and correction pathways differ (Barros, 4 Mar 2025).

3. Multimodal, agentic, and embodied manifestations

In multimodal reasoning models, longer reasoning chains often weaken perceptual grounding rather than strengthen it. “More Thinking, Less Seeing?” reports that as generations become longer, models allocate less attention to visual tokens and more to instruction and language tokens, leading to drift away from image-grounded content. The paper introduces RH-AUC to quantify the balance between reasoning performance and perceptual fidelity across reasoning lengths, and finds a non-monotonic trade-off: moderate-length reasoning can be optimal, whereas excessive reasoning degrades both grounding and accuracy (Liu et al., 23 May 2025).

This multimodal phenomenon is closely related to the distinction between thinking while looking and thinking before looking. “Thinking Before Looking” argues that standard multimodal CoT tightly entangles image and text during reasoning, encouraging stereotypical reasoning patterns and visual-prior bias. Its diagnosis is that simultaneous access to the image during reasoning can aggravate reasoning-related hallucination instead of mitigating it (Zheng et al., 2024).

In tool-using agents, the same pattern appears in a different form. SimpleToolHalluBench isolates two failure modes—No Tool Available (NTA) and Distractor Tool (DT)—and shows that progressively enhancing reasoning through RL increases tool hallucination proportionally with task performance gains. The effect is method-agnostic: it appears after RL, after supervised fine-tuning or distillation, and even when reasoning is elicited only at inference. For Qwen3-8B, simply enabling thinking changes $R_{\text{NTA}}$ from 4.1% to 5.4% and $R_{\text{DT}}$ from 36.2% to 56.8% (Yin et al., 27 Oct 2025).

Embodied agents exhibit a grounding-specific variant. HEAL defines hallucination as generation of plans, object references, or state references not grounded in the observed physical environment, and constructs a probing set that can induce hallucination rates up to 40x higher than base prompts. The most severe failures occur under Scene Task Contradiction, where the instruction requires absent objects and the planner reasons as if they exist. The paper emphasizes that these are not merely copying errors: reasoning traces explicitly rationalize why a missing object “should” exist and proceed accordingly (Chakraborty et al., 18 Jun 2025).

Video understanding adds temporal complexity. HAVEN organizes hallucination in video LMMs along hallucination causes, hallucination aspects, and question formats, yielding about 6K questions. It shows that CoT often improves accuracy and usually reduces bias, but the relationship is not uniform across models; for LLaMA-VID-7B, CoT improves accuracy while increasing bias (Gao et al., 25 Mar 2025). Adversarial image generation reveals an even more direct attack on the reasoning process: GHOST generates natural-looking, object-free images that reroute MLLM reasoning, achieving a hallucination success rate of about 32.4% on GLM-4.1V-Thinking while keeping the target object absent (Parast et al., 29 Sep 2025).

4. Evaluation and diagnosis beyond final-answer accuracy

Because thinking-induced hallucination often resides in the reasoning trace rather than the final answer, recent evaluation frameworks explicitly score intermediate cognition, grounding drift, or abstention failures.

Framework	Setting	Main diagnostic focus
RACE (Wang et al., 5 Jun 2025)	LRMs	Inter-sample reasoning consistency, answer uncertainty, reasoning-answer alignment, internal coherence
MM-THEBench (Huang et al., 30 Jan 2026)	Reasoning MLLMs	Step-level hallucinations in perception, knowledge, and reasoning; intermediate CoT correctness lower than final answer accuracy
RH-Bench / RH-AUC (Liu et al., 23 May 2025)	Multimodal reasoning length	Change in perceptual accuracy as reasoning length varies
SimpleToolHalluBench (Yin et al., 27 Oct 2025)	Tool-using agents	Hallucination when no correct tool exists or only distractor tools are available
HAVEN (Gao et al., 25 Mar 2025)	Video understanding	Hallucination causes, aspects, and question formats; accuracy and bias score
HEAL probing set (Chakraborty et al., 18 Jun 2025)	Embodied planning	Scene-task inconsistency, measured with CHAIR and POPE
HIP/HQP (Sato, 1 May 2025)	Prompt-induced hallucination	Hallucination score from 0 to 10 under semantically unstable conceptual fusion

These frameworks converge on a common result: answer-level correctness is not a sufficient proxy for faithful reasoning. RACE explicitly targets cases where sampled answers look consistent while their chains of thought are contradictory, unsupported, or generic, and reports that inter-reasoning consistency contributes the largest single-signal improvement over answer-only baselines (Wang et al., 5 Jun 2025). MM-THEBench likewise reports that intermediate CoT correctness is significantly lower than final answer accuracy, and that reasoning hallucinations are more strongly correlated with incorrect final answers than pure perception errors (Huang et al., 30 Jan 2026).

A second diagnostic theme is abstention. In HEAL, the correct behavior under Task Relevant Object Removal and Scene Task Contradiction is not creative substitution but refusal, rejection, or an empty plan (Chakraborty et al., 18 Jun 2025). SimpleToolHalluBench makes the same point in agentic form: trustworthy reasoning includes knowing when no legitimate tool action exists (Yin et al., 27 Oct 2025).

5. Mitigation strategies and their trade-offs

One major strategy is to inject grounded structure before or during reasoning. Re-Critic augments multimodal instruction tuning with a Visual Rationale Synthesizer (VCIT) and an in-context self-critic mechanism for preference tuning. The rationale is inserted into the input prompt rather than emitted as output, and the self-critic selects better response pairs for DPO. On hallucination benchmarks, the method improves average performance by +6.2% for LLaVA-v1.5 (7B) and +2.7% for InternVL 2.0 (2B); with LLaVA-80K, it yields a 31.8% improvement over baseline. On Object HalBench for LLaVA-v1.5, response-level hallucination decreases from 56.4% to 49.6% and mention-level hallucination from 27.9% to 24.9%. The same model family also improves on MathVista, indicating that the gains extend beyond hallucination-specific tasks (Yang et al., 12 May 2025).

A second line uses counterfactual or contrastive prompting at inference time. Counterfactual Inception generates plausible but absent keywords, filters them by CLIP similarity and NLI contradiction score, and re-prompts the model so that it reasons against carefully chosen alternatives. Across models, this reduces hallucination without retraining; in human consistency scoring, average hallucinatory keywords per image decrease from 1.7 to 0.81, while consistency increases from 3.3 to 4.13 (Kim et al., 2024). VIC follows a different route: it builds a reasoning chain from textual context alone before introducing the image. On MMVP, Gemini 1.5 Pro improves from 0.420 to 0.553, and on HallusionBench, GPT-4o improves from 0.626 to 0.692 (Zheng et al., 2024).

A third family regulates when thinking should occur at all. MARGO trains with mixed-mode rollout groups containing both thinking and non-thinking trajectories, rewarding reasoning only when it adds factual value beyond direct answering. It raises average accuracy from 22.81% to 25.49% on Qwen3-4B and from 27.59% to 29.57% on Qwen3-8B, while preserving mathematical reasoning ability on AMC23, AIME24, and AIME25 (Wang et al., 7 Jul 2026). HaluSearch frames generation as tree search with a self-evaluation reward model and a fast/slow thinking switch; on HaluEval-QA with Llama3.1-8B-Instruct, direct inference attains 35.6%, whereas MCTS (HaluSearch) reaches 45.4% (Cheng et al., 2 Jan 2025).

Mitigation remains domain-dependent and often costly. In video LMMs, SRFT plus TDPO improves hallucination accuracy by 7.65% over the baseline and reduces the bias score by 4.5% on HAVEN (Gao et al., 25 Mar 2025). In embodied planning, Self-Correcting Woodpecker (SCW) outperforms Knowledge-Augmented Feedback (KAF), but hallucination remains high in the hardest SceneTaskCon setting (Chakraborty et al., 18 Jun 2025). For tool hallucination, DPO can reduce $R_{\text{NTA}}$ from 90.2% to 55.8% and $R_{\text{DT}}$ from 100.0% to 71.4%, yet SynTool Reward drops from 0.45 to 0.34, making the reliability-capability trade-off explicit (Yin et al., 27 Oct 2025). GHOST shows that adversarially generated negative examples can also be used constructively: fine-tuning Qwen2.5-VL on GHOST images reduces hallucination on unseen GHOST-LLaVA images from 52.6% to 7.0%, on GHOST-GLM from 63.1% to 7.3%, while POPE rises from 88.5 to 90.7 (Parast et al., 29 Sep 2025).

6. Human processing, theory, and contested implications

The study of thinking-induced hallucination is not limited to machine behavior. A neuroimaging study using EEG and ERP analysis records signals from 27 participants performing image-description verification on MLLM-generated captions. It finds that correctly recognized hallucinations elicit enhanced N100, P200, N400, and P600 responses, whereas hallucinations that are misjudged as correct are neurally indistinguishable from accurate descriptions: HalluWrong vs. NoHallu shows no statistically significant differences in any ERP component or region. EEG-based prediction achieves AUC > 0.9 within subject only when participants recognize the hallucination; when they are misled, performance falls to chance. The paper interprets this as failure to trigger the standard neurocognitive fact-verification pathway (Zhu et al., 16 May 2026).

At a broader theoretical level, predictive-processing accounts treat hallucination as a consequence of advanced inference under uncertainty. One recent comparison argues that both humans and LLMs are predictive systems that interpolate or extrapolate missing information; in humans this occurs through hierarchical prediction and prediction-error correction, while in LLMs it occurs through auto-regressive next-token modeling without intrinsic truth checking (Barros, 4 Mar 2025). A very different and explicitly speculative line models hallucination in schizophrenia as desynchronized time evolution of percept basis states in an infinite-dimensional Hilbertian consciousness, proposing that thought-driven activation of perceptual basis vectors can generate hallucinatory experience when internal evolution is out of sync with real-world time (1706.03619).

The normative status of hallucination is also contested. HIC-Bench argues that reducing hallucination indiscriminately may suppress creative potential, and reports a nonlinear relationship between IH and DH rather than a simple antagonism. Under DHP + RCP, GPT-4o reaches an IH ratio of 21.1% with DH down to 0.3%, and the automated IH/DH labeling is supported by human verification with precision and recall above 85% for both classes (Yang et al., 25 Dec 2025). This does not negate the harms of defective hallucination, but it complicates any definition that equates all evidence-exceeding generation with defect.

Taken together, these studies undermine three recurring assumptions. More reasoning does not reliably produce more truth; correct final answers do not guarantee faithful intermediate reasoning; and hallucination is not always a single-category pathology. Current work therefore treats thinking-induced hallucination as an interaction between reasoning depth, grounding, metacognition, prompting, and task structure. The unresolved problem is how to preserve the benefits of explicit thinking while preventing the reasoning process itself from becoming an independent source of error.