Failing to Falsify: Evaluating and Mitigating Confirmation Bias in Language Models

Abstract: Confirmation bias, the tendency to seek evidence that supports rather than challenges one's belief, hinders one's reasoning ability. We examine whether LLMs exhibit confirmation bias by adapting the rule-discovery study from human psychology: given a sequence of three numbers (a "triple"), an agent engages in an interactive feedback loop where it (1) proposes a new triple, (2) receives feedback on whether it satisfies the hidden rule, and (3) guesses the rule. Across eleven LLMs of multiple families and scales, we find that LLMs exhibit confirmation bias, often proposing triples to confirm their hypothesis rather than trying to falsify it. This leads to slower and less frequent discovery of the hidden rule. We further explore intervention strategies (e.g., encouraging the agent to consider counter examples) developed for humans. We find prompting LLMs with such instruction consistently decreases confirmation bias in LLMs, improving rule discovery rates from 42% to 56% on average. Lastly, we mitigate confirmation bias by distilling intervention-induced behavior into LLMs, showing promising generalization to a new task, the Blicket test. Our work shows that confirmation bias is a limitation of LLMs in hypothesis exploration, and that it can be mitigated via injecting interventions designed for humans.

• The paper quantifies confirmation bias in LLMs using a controlled, Wason-inspired rule-discovery framework that measures incompatible-to-compatible test ratios.
• It introduces human-inspired interventions like Dual-Goal and Think-in-Opposites, which significantly reduce confirmatory testing and improve rule discovery rates.
• The study further demonstrates that debiasing policies can be internalized via knowledge distillation, allowing for effective cross-domain generalization in LLM reasoning.

Evaluating and Mitigating Confirmation Bias in LLMs

Introduction

This paper investigates confirmation bias in LLMs, specifically their tendency to seek confirmatory evidence over falsificatory evidence during hypothesis-driven exploration. Drawing on paradigms from human cognitive psychology—most notably, the Wason 2-4-6 rule discovery task—the authors present a controlled framework for quantifying confirmation bias in LLMs, evaluate its prevalence across multiple model families and inference settings, and develop intervention strategies to mitigate its effects. Confirmation bias is operationalized as a reduced rate of incompatible (potentially falsifying) tests relative to compatible (confirming) ones, mirroring cognitive diagnostics in human subjects.

Experimental Setup

A synthetic rule-discovery environment inspired by Wason's paradigm is established, in which a model receives a triple of integers conforming to a hidden rule. Models alternate between hypothesizing about the rule and proposing new test triples, receiving binary feedback on whether tests conform to the rule. The process is repeated over up to 45 turns per episode, and evaluation metrics include task success rate (correct rule identification), efficiency (turns to success), and the incompatible-to-compatible proposal ratio (I:C), a process-level measure of confirmation bias.

Eleven LLMs are assessed, spanning both non-thinking and reasoning-tracing (thinking mode) variants. The data includes artificially generated and human-extracted relational rules over integer triples, enabling structured multi-task evaluation and domain generalization testing.

Confirmation Bias in LLMs

Across all model families, significant confirmation bias is observed: most LLMs preferentially propose test triples that are compatible with their current hypotheses. As a result, they discover underlying rules more slowly and less frequently, exhibiting lower task success rates compared to more balanced explorative approaches. Notably, larger-scale models and those employing explicit reasoning traces ("thinking mode") tend to achieve higher task success rates and display less confirmation bias, but the effect is not entirely explained by scale or architecture alone.

Crucially, a strong positive correlation (Spearman ρ ≈ 0.54–0.75, $p < 0.05$) is found between the I:C ratio and task success. This pattern converges with human cognitive studies: agents (human or model) exhibiting less confirmation bias perform better in hypothesis discovery tasks.

Intervention Strategies

The authors test two psychologically-motivated interventions, adapted from human debiasing literature:

• Dual-Goal: Models are instructed to simultaneously hypothesize the rule and its complement, encouraging exploration of both confirming and disconfirming evidence.
• Think-in-Opposites (TiO): Models are prompted to, at each test, generate a triple that is opposite along a salient dimension of their current hypothesis.

Both interventions yield statistically significant reductions in confirmation bias and improvements in discovery rates for most "thinking mode" models. For example, average rule discovery rates increase from 42% to 56%, and compatible test proposals are proportionally reduced. However, effects in non-thinking models are mixed: TiO yields a modest increase in accuracy, whereas Dual-Goal sometimes reduces accuracy, suggesting the effectiveness of these strategies is architecture-dependent.

Internalization via Knowledge Distillation

To move beyond inference-time prompts, the paper explores distilling these intervention-induced exploration policies directly into LLM weights. Using supervised fine-tuning, student models are trained to imitate the test proposal behavior from intervention-guided "teacher" runs. Results indicate that intervention distillation notably increases both task success rates and the I:C ratio, often matching or exceeding teacher performance. A cross-scale distillation strategy—training a smaller model on a larger, intervention-guided teacher—produces substantial gains, nearly tripling baseline task success in some configurations and more than doubling incompatible testing.

Generalization to Novel Domains

The transferability of debiasing is evaluated in the Blicket Test, an object-based causal discovery task adapted from developmental psychology. Models fine-tuned for lower confirmation bias on the Wason-style number task generalize successfully, exhibiting increased incompatible testing and improved task success rates in the Blicket domain. Notably, the positive I:C/success correlation persists, indicating that reduced confirmation bias in one task translates to improved falsification-oriented reasoning in others. However, in-domain prompting on the new task still yields superior gains, suggesting partial, but not complete, generalization.

Theoretical and Practical Implications

These findings extend existing LLM bias research from evaluation settings (stance preference) to exploration settings, showing that major LLMs are not only biased in what evidence they prefer to accept, but in what evidence they actively seek when reasoning interactively. As autonomous LLM agents are increasingly used in scientific discovery, strategic planning, and other hypothesis-rich domains, confirmation bias in exploration constitutes a consequential failure mode: it leads to inefficient search and can severely hinder scientific or diagnostic tasks.

The demonstrated efficacy of human-inspired debiasing intervention—especially when internalized via distillation and validated to generalize—meaningfully advances LLM alignment. It suggests that targeted interventions originally developed for humans can be leveraged for LLMs, that these interventions are learnable as policies rather than one-off instructions, and that cognitive process-level bias metrics are operational in LLMs.

Future Directions

Key open questions include whether confirmation bias reduction generalizes to more naturalistic or less-structured reasoning domains, to problems with sparse or noisy feedback, or to longer-term autonomous agent settings. The relative roles of architectural scaling, explicit trace prompting, and distillation protocols also warrant further study. Finally, the framework and metrics developed here provide useful baselines for evaluating future models' hypothesis exploration behavior and for driving the design of more genuinely scientific LLMs.

Conclusion

This paper provides systematic evidence that LLMs inherit the confirmation bias of humans in hypothesis-space exploration and establishes that both prompt-based and learned intervention strategies, originally developed in cognitive psychology, can quantitatively mitigate this bias. By connecting process-level metrics of hypothesis testing to concrete performance improvements and demonstrating cross-domain transfer, this work offers both a diagnostic framework and actionable pathways for the development of more robust and unbiased LLM agents.

Reference: "Failing to Falsify: Evaluating and Mitigating Confirmation Bias in LLMs" (2604.02485)