Cause of the false confidence phenomenon

Determine the precise mechanisms and conditions that cause false confidence in probabilistic uncertainty quantification procedures—such as Bayesian and fiducial posteriors—where data-dependent probabilities assign high confidence to false hypotheses, and characterize this unreliability across statistical models and hypothesis classes.

Background

The paper establishes that any data-dependent posterior probability (Bayes with default priors, fiducial, etc.) can exhibit false confidence, meaning high posterior probability is assigned to false hypotheses. This unreliability is formalized by the false confidence theorem and illustrated with examples.

While the phenomenon is demonstrated, the underlying cause remains unidentified. The author notes that existing analyses suggest nonlinearity (e.g., hypotheses about ratios) may be implicated, but no general characterization has been proven.