Overconfidence Phenomenon in AI

Updated 17 July 2026

Overconfidence Phenomenon is the systematic excess of subjective certainty over empirical evidence, manifesting in miscalibrated judgments and biased decision-making.
It is observed through phenomena such as inflated verbal confidence in LLMs, under-covered numerical intervals, and overrated task-success probabilities in agentic systems.
Mitigation strategies include recalibrating reward models, applying post hoc statistical adjustments, and refining model components to better align confidence with actual performance.

Overconfidence denotes a systematic excess of subjective certainty over warranted accuracy, correctness, completeness, or evidential support. Across the literature summarized here, the term does not refer to a single invariant construct: it appears as self-deception about one’s own capability in evolutionary games, as miscalibration between stated confidence and realized success in human and machine judgment, as excessive verbal certainty in LLMs, as too-narrow confidence intervals in numerical estimation, as structurally induced certainty in out-of-distribution and open-set recognition, and as an apparent but sometimes physically induced posterior collapse in Bayesian scientific machine learning (Li et al., 2016, Leng et al., 2024, Epstein et al., 30 Oct 2025, Landgren et al., 28 Sep 2025). A unifying characterization is that some internal or external estimate of reliability is sharper, larger, or more complete-seeming than the underlying evidence justifies.

1. Conceptual scope and major forms

The overconfidence phenomenon is operationalized differently across research areas.

Domain	Operationalization	Representative source
RLHF-aligned LLMs	Verbalized confidence exceeds response quality or correctness	(Leng et al., 2024)
Numerical uncertainty	Nominal confidence intervals under-cover true answers	(Epstein et al., 30 Oct 2025)
Agentic systems	Predicted task-success probability exceeds realized success	(Kaddour et al., 6 Feb 2026)
Scholarly assistance	Assertive, complete-seeming output under incomplete or unverifiable knowledge	(Li et al., 6 Jun 2026)
Multi-solution reasoning	Undue certainty in an incomplete solution set	(Guan et al., 1 Dec 2025)
Evolutionary behavior	Self-assessed capability exceeds real capability	(Li et al., 2016)

In calibration-centered work, overconfidence is a misalignment between confidence and empirical correctness or quality. The RLHF study defines calibration as alignment between model confidence and actual performance, then distinguishes token-level sharpening from a second phenomenon—verbalized overconfidence—in which a model explicitly states excessive certainty in natural language (Leng et al., 2024). Fermi-style numerical estimation casts the same problem in interval form: a model that claims $99\%$ confidence but covers the truth far less often is overconfident in the classical probabilistic sense (Epstein et al., 30 Oct 2025). Agentic work generalizes the target from single predictions to entire trajectories through $P(\mathrm{IS})$ , the probability that an agent built on the same model succeeds on a task given the current information state (Kaddour et al., 6 Feb 2026).

Other papers widen the concept beyond explicit calibration. GIScholarBench defines overconfidence behaviorally as the production of confident, assertive, well-formatted scholarly outputs even when the required knowledge is incomplete or unverifiable, rather than through elicited confidence scores (Li et al., 6 Jun 2026). MuSoBench relocates the phenomenon from answer correctness to solution completeness: a model may be locally correct yet overconfident about having exhausted the solution space (Guan et al., 1 Dec 2025). In evolutionary game theory, overconfidence is not verbal certainty at all, but a state variable $\alpha_i$ added to real capability $\gamma_i$ , producing self-perceived capability $k_i=\gamma_i+\alpha_i$ (Li et al., 2016).

A further complication is that “overconfidence” can be partly structural. In Bayesian PINNs, narrow posterior bands may reflect either genuine restriction by physical constraints or an artifact of posterior collapse. The paper therefore argues that standard overconfidence language is too coarse unless one distinguishes confidence justified by the physics prior from confidence unsupported by information density (Landgren et al., 28 Sep 2025). This suggests that overconfidence is best treated as a family of phenomena linked by excess certainty, but differentiated by the object of certainty: correctness, interval coverage, task success, completeness, or epistemic support.

2. Formalization and measurement

The most common formalization is bin-based calibration error. For RLHF-calibrated LLMs, Expected Calibration Error and AUC are primary metrics; in the paper’s wording, ECE bins predictions by confidence and compares accuracy within bins, while AUC measures the ability of confidence to separate correct from incorrect responses (Leng et al., 2024). MuSoBench adapts the same logic to completeness by computing confidence-relative calibration against Recall rather than only Precision. Its definitions include

$\text{Recall}(x)=\frac{|\mathcal{Y}_i \cap \hat{\mathcal{Y}}_i|}{|\hat{\mathcal{Y}}_i|}$

and

$\text{ECE(r)}=\sum_{m=1}^{M}\frac{|B_m|}{N}\,|\text{recall}(B_m)-\text{conf}(B_m)|,$

thereby measuring whether confidence tracks recovery of the full solution set (Guan et al., 1 Dec 2025).

For interval-valued uncertainty, FermiEval distinguishes nominal confidence $1-\alpha$ from observed coverage. The empirical estimator is

$\widehat{\mathrm{coverage}}=\frac{1}{N}\sum_{i=1}^N \mathbf{1}[y_i\in [L_i,U_i]],$

and a calibrated model should satisfy $\widehat{\mathrm{coverage}}\approx 1-\alpha$ . To combine coverage with sharpness, the benchmark uses the Winkler interval score,

$P(\mathrm{IS})$ 0

so that narrow but badly under-covering intervals are penalized (Epstein et al., 30 Oct 2025).

Several papers use simpler bias measures. In reasoning tasks with known ground truth, overconfidence is

$P(\mathrm{IS})$ 1

where $P(\mathrm{IS})$ 2 is stated probability of being correct and $P(\mathrm{IS})$ 3 is actual correctness (Sun et al., 4 May 2025). Agentic overconfidence uses the same directional logic at task level: mean predicted success probability minus the true base success rate for that condition (Kaddour et al., 6 Feb 2026). Human confidence learning work also treats miscalibration as the gap between ex ante subjective confidence and ex post success frequency, though its main emphasis is on dynamics rather than a single summary index (Lévy-Garboua et al., 2017).

Judge systems motivate a deployment-oriented metric. The LLM-as-a-Judge paper introduces TH-Score,

$P(\mathrm{IS})$ 4

computed over specified high- or low-confidence intervals to emphasize the parts of the confidence distribution that matter for automatic acceptance or review routing (Tian et al., 8 Aug 2025). This differs from ECE, ACE, Brier score, MCE, and NLL by explicitly combining interval accuracy with interval coverage for risk-aware selective automation.

Not all work accepts calibration metrics as sufficient. Bayesian PINNs argue that reliability diagrams, prediction-interval coverage, PIT histograms, and related tools conflate data-driven and physics-driven posterior contraction. Their alternative diagnostics are information density

$P(\mathrm{IS})$ 5

and global physics-constraint coupling

$P(\mathrm{IS})$ 6

which are intended to distinguish justified narrowing caused by strong physical constraints from artifactual overconfidence (Landgren et al., 28 Sep 2025).

3. Generative mechanisms

The surveyed work identifies multiple, partly incompatible mechanisms.

One class is biased evidence aggregation. In human confidence learning, uncertainty about one’s own ability induces contrarian illusory signals generated by doubt. For likely success, doubt makes failure salient and depresses confidence; for likely failure, doubt makes success salient and inflates confidence. The model writes these as

$P(\mathrm{IS})$ 7

and uses this mechanism to account jointly for the hard–easy effect, Dunning–Kruger-type ability gradients, conservative learning, and overprecision (Lévy-Garboua et al., 2017).

A second class is objective-induced optimization bias. In RLHF, reward models used during PPO are shown to prefer higher-confidence versions of the same response, including rejected ones, so confidence acts as a spurious reward feature. This makes the policy optimize toward confident-sounding language even when quality does not warrant it (Leng et al., 2024). In sequential recommendation, negative sampling similarly distorts the effective class prior, yielding the BCE equilibrium

$P(\mathrm{IS})$ 8

which pushes predicted probabilities for top items toward one and produces overconfidence in SASRec-style training (Petrov et al., 2023).

A third class is search or inference collapse. MuSoBench proposes the cognitive-rigidity hypothesis, according to which overconfidence arises when the reasoning process prematurely converges on a narrow set of thought paths, producing high certainty in a partial solution set (Guan et al., 1 Dec 2025). FermiEval proposes a related but distributional account, the perception-tunnel theory: on each query, the model behaves as if it reasons over a truncated portion $P(\mathrm{IS})$ 9 of its inferred uncertainty distribution rather than the full $\alpha_i$ 0, leading to tail neglect and overly narrow intervals (Epstein et al., 30 Oct 2025).

A fourth class is post hoc confirmation and anchoring. Agentic overconfidence in coding agents is attributed partly to confirmation bias in post-execution review: once a coherent-looking patch exists, reviewers mistake plausibility for correctness. This helps explain the paper’s counterintuitive result that pre-execution assessment can discriminate success from failure better than vanilla post-execution review (Kaddour et al., 6 Feb 2026).

A fifth class is architectural geometry. In OOD detection, prior theory for piecewise-linear ReLU networks implies that as certain inputs are scaled away from the training distribution, the softmax can converge toward a one-hot vector. The CEA paper treats overconfidence here as intrinsic to extrapolation geometry and proves that asymptotic softmax overconfidence with a linear final layer implies divergence in at least one penultimate activation coordinate (Azizmalayeri et al., 2024). Open-set recognition work traces a related phenomenon to feature-space overlap between unknown classes and known-class manifolds, producing unjustifiably high confidence on semantically similar unknowns (Zhao et al., 15 Nov 2025).

Finally, some work argues that apparent overconfidence may be adaptive or induced by constraints rather than purely erroneous. In Bayesian PINNs, hard or soft physics constraints can legitimately collapse posterior variance. In organizational theory, overconfidence can be selected because it amplifies responsiveness to private information when conflicts of interest vary across states; a well-calibrated employee is optimal iff $\alpha_i$ 1 is constant (Espitia, 8 Jan 2026).

4. Empirical manifestations in contemporary AI

LLMs exhibit the phenomenon across multiple levels of abstraction. After RLHF, confidence distributions shift upward, and models verbally express excessive certainty even when wrong. On Llama3-8B, GSM8K, CoT prompting, vanilla PPO achieves ECE $\alpha_i$ 2, while PPO-M reduces it to $\alpha_i$ 3 and raises accuracy from $\alpha_i$ 4 to $\alpha_i$ 5; the paper frames this as a 6.44-point ECE reduction and a 2.73-point accuracy gain relative to standard PPO (Leng et al., 2024). Mechanistic work later localized inflated verbalized confidence to a compact set of middle-to-late-layer MLPs and attention heads, with targeted steering cutting PopQA ECE from $\alpha_i$ 6 to $\alpha_i$ 7 on Qwen2.5-3B and from $\alpha_i$ 8 to as low as $\alpha_i$ 9 on Llama-3.2-3B, depending on the intervention (Zhao et al., 1 Apr 2026).

Numerical uncertainty remains sharply miscalibrated. FermiEval reports that across several modern models, nominal $\gamma_i$ 0 intervals cover the truth only about $\gamma_i$ 1 of the time on average, with conformal adjustment restoring accurate $\gamma_i$ 2 observed coverage and decreasing the Winkler score by $\gamma_i$ 3 (Epstein et al., 30 Oct 2025). On reasoning problems with known ground truths, all five studied LLMs overestimate the probability that their answer is correct by between 20% and 60%; GPT-4o, for example, has fraction correct $\gamma_i$ 4, mean confidence $\gamma_i$ 5, and bias $\gamma_i$ 6, whereas humans on the same paradigm are only about 4 percentage points overconfident on average (Sun et al., 4 May 2025).

Task-level and interactive settings intensify the problem. In coding-agent evaluation, some agents that succeed only 22% of the time predict 77% success, and post-execution review can be worse calibrated than pre-execution assessment. GPT-5.2-Codex post-execution reviewers, for instance, predict 73% success against a true rate of 35% (Kaddour et al., 6 Feb 2026). In multi-turn debate, the zero-sum structure makes overconfidence logically transparent: initial confidence averages $\gamma_i$ 7 against a rational $\gamma_i$ 8 baseline, rises to $\gamma_i$ 9 by the final round, and in 61.7% of cross-model debates both sides simultaneously claim at least $k_i=\gamma_i+\alpha_i$ 0 probability of victory (Prasad et al., 25 May 2025).

Broader AI evaluation pipelines show analogous failures. GIScholarBench finds consistent overconfidence across metadata retrieval, literature linking, and research-direction generation: models continue producing definitive titles and DOIs when wrong, extend citation lists beyond reliable retrieval capacity, and generate future research agendas with lower topic coverage and higher novel miss rates than real future-citing papers (Li et al., 6 Jun 2026). In LLM-as-a-Judge systems, the concern is not only wrong verdicts but inflated confidence on those verdicts, motivating a shift from accuracy-centric to confidence-driven, risk-aware judge design (Tian et al., 8 Aug 2025).

Beyond LLMs, the same pattern appears in uncertainty-sensitive ML. In OOD detection, AUC can degrade as inputs are pushed farther from the in-distribution; for eICU with ResNet plus MSP, AUC falls from 47.9 at scaling factor 10 to 13.2 at 1000, a reversal of the intended novelty ranking (Azizmalayeri et al., 2024). Open-set recognition similarly documents high-confidence known-class assignments on semantically similar unknowns, while active learning shows that biased small-batch updates induce overconfident false predictions that in turn corrupt uncertainty-based selection (Zhao et al., 15 Nov 2025, Hwang et al., 2023).

In human cognition, overconfidence is dynamic rather than static. The double-or-quits experiment shows that participants are underconfident on an easy training task but overconfident on harder future tasks, and that they learn to be overconfident faster than they learn their true ability (Lévy-Garboua et al., 2017). In a distinct perceptual setting, microgravity increases subjective confidence without corresponding performance gains, especially under greater stimulus uncertainty; the paper interprets this as a confidence–performance dissociation consistent with confidence inflation rather than a full demonstration of metacognitive miscalibration (Loued-Khenissi et al., 2023).

Evolutionary models treat overconfidence as socially embedded self-deception. In the resource competition game, each player has real capability $k_i=\gamma_i+\alpha_i$ 1, overconfidence intensity $k_i=\gamma_i+\alpha_i$ 2, and bluffing intensity $k_i=\gamma_i+\alpha_i$ 3, with self-perceived capability $k_i=\gamma_i+\alpha_i$ 4 and displayed capability $k_i=\gamma_i+\alpha_i$ 5. Across structured populations, bluffing generally evolves to higher levels than overconfidence, but both persist because they can increase access to contested resources, especially when true ability is imperfectly observed (Li et al., 2016). Network topology matters: heterogeneity facilitates bluffing while improving the efficiency of adequate punishment against overconfident behavior (Li et al., 2016).

A related evolutionary game on spatial prisoner’s dilemma dynamics shifts the object of bias from capability to payoff comparison. Overconfident players overestimate their own payoff as $k_i=\gamma_i+\alpha_i$ 6, making them less likely to imitate neighbors. This suppresses both cooperative and defective invasions, but defective spread is hindered more strongly, reinforcing spatial reciprocity and helping maintain a “reciprocity-based cooperative phalanx” (Szolnoki et al., 2018). Here overconfidence is not epistemically beneficial, but it has a collective side effect favorable to cooperation under nonlinear imitation dynamics.

Organizational theory introduces a different normative angle. In delegated decision-making under private information and conflict of interest, the organization may prefer a miscalibrated employee because confidence shapes how strongly signals are acted on. The paper’s central result is that an employee is optimally well-calibrated iff the expected conflict $k_i=\gamma_i+\alpha_i$ 7 is constant across signals; when stronger responsiveness is desired, overconfidence becomes optimal (Espitia, 8 Jan 2026). This does not reclassify overconfidence as accurate. Rather, it treats it as a potentially useful distortion in environments where action sensitivity matters more than unbiased belief formation.

6. Mitigation strategies and unresolved questions

The most direct interventions modify the confidence-generating process itself. In RLHF, PPO-M calibrates the reward model so that high confidence is rewarded on chosen responses and low confidence on rejected responses, while PPO-C adjusts the online reward by comparing reward-free quality to a running threshold. Both reduce ECE while preserving performance, and the same logic extends to DPO through CDPO (Leng et al., 2024). Mechanistic work achieves inference-time recalibration by ablating or steering a small set of confidence-writing components rather than retraining the full model (Zhao et al., 1 Apr 2026).

Post hoc statistical calibration is also effective. FermiEval finds split conformal prediction to be the strongest interval-repair method, restoring near-target coverage and sharply improving Winkler score, while direct log-probability elicitation provides a weaker but lightweight baseline when calibration data are unavailable (Epstein et al., 30 Oct 2025). In agentic systems, reframing evaluation as bug-finding rather than correctness confirmation reduces ECE and overconfidence across all studied coding-agent families, and conservative ensembling between pre- and post-execution estimates further improves calibration in some cases (Kaddour et al., 6 Feb 2026). For judges, the proposed remedy is LLM-as-a-Fuser, coupled with TH-Score and a confidence-driven pipeline, to make automated evaluation adaptive rather than purely accuracy maximizing (Tian et al., 8 Aug 2025).

Outside LLMs, several methods attack structural sources of overconfidence. CEA augments any OOD novelty score with a penalty derived from extreme penultimate activations, often converting catastrophically poor AUC into strong detection performance in ReLU-based models (Azizmalayeri et al., 2024). In sequential recommendation, gBCE and gSASRec correct negative-sampling-induced probability inflation while improving top-rank recommendation quality (Petrov et al., 2023). In active learning, CMaM expands the effective training distribution through dual-space mixing, while RankedMS alters acquisition to avoid selection driven by distorted confidence scores (Hwang et al., 2023). Open-set recognition uses perturbation-based uncertainty estimation plus a two-stage detector to mitigate high-confidence misclassification of semantically similar unknowns (Zhao et al., 15 Nov 2025). For multi-solution reasoning, Long-CoT, reflection, explicit exploration prompts, and voting-based self-consistency all improve recall and reduce reasoning overconfidence, whereas higher decoding temperature alone does not (Guan et al., 1 Dec 2025).

Several open problems recur. One is the relation between verbalized confidence and deeper latent uncertainty: RLHF, mechanistic LLM work, and GIS-style behavioral benchmarks all indicate that polished certainty can decouple from correctness, but the exact internal interfaces remain only partly understood (Leng et al., 2024, Zhao et al., 1 Apr 2026, Li et al., 6 Jun 2026). Another is whether apparent overconfidence is always a bug: Bayesian PINNs argue that some narrow uncertainty is physically warranted, not pathological (Landgren et al., 28 Sep 2025). A third is coverage beyond scalar correctness. Multi-solution tasks and scholarly ideation show that future work must evaluate not only whether answers are right, but whether search is complete, citation neighborhoods are responsibly bounded, and ideation adequately covers the relevant semantic space (Guan et al., 1 Dec 2025, Li et al., 6 Jun 2026).

Taken together, these results support a broad but technically precise conclusion: overconfidence is best understood as an excess-certainty failure mode whose observable form depends on what the system is claiming—correctness, interval coverage, task success, out-of-distribution familiarity, citation support, or solution completeness. Its causes range from doubt-induced belief distortion to optimization bias, architectural geometry, confirmation bias, premature search convergence, and strategic or stylistic answer formation. Its mitigation therefore requires interventions at multiple levels: objectives, reward models, uncertainty estimators, search procedures, mechanistic circuits, and evaluation protocols.