Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
120 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Which statistical hypotheses are afflicted with false confidence? (2404.16228v1)

Published 24 Apr 2024 in math.ST and stat.TH

Abstract: The false confidence theorem establishes that, for any data-driven, precise-probabilistic method for uncertainty quantification, there exists (non-trivial) false hypotheses to which the method tends to assign high confidence. This raises concerns about the reliability of these widely-used methods, and shines new light on the consonant belief function-based methods that are provably immune to false confidence. But an existence result alone is insufficient. Towards a partial answer to the title question, I show that, roughly, complements of convex hypotheses are afflicted by false confidence.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (37)
  1. Balch, M. S. (2012). Mathematical foundations for a theory of confidence structures. Internat. J. Approx. Reason., 53(7):1003–1019.
  2. Satellite conjunction analysis and the false confidence theorem. Proc. Royal Soc. A, 475(2227):2018.0565.
  3. An exposition of the false confidence theorem. Stat, 7(1):e201.
  4. Confidence in confidence distributions! Proc. Roy. Soc. A, 476:20190781.
  5. Dawid, A. P. (2020). Fiducial inference then and now. arXiv:2012.10689.
  6. Dempster, A. P. (1966). New methods for reasoning towards posterior distributions based on sample data. Ann. Math. Statist., 37:355–374.
  7. Dempster, A. P. (1968). A generalization of Bayesian inference. (With discussion). J. Roy. Statist. Soc. Ser. B, 30:205–247.
  8. Dempster, A. P. (2008). The Dempster–Shafer calculus for statisticians. Internat. J. Approx. Reason., 48(2):365–377.
  9. Denœux, T. (2014). Likelihood-based belief function: justification and some extensions to low-quality data. Internat. J. Approx. Reason., 55(7):1535–1547.
  10. Frequency-calibrated belief functions: review and new insights. Internat. J. Approx. Reason., 92:232–254.
  11. Possibility Theory. Plenum Press, New York.
  12. Fisher, R. A. (1930). Inverse probability. Proceedings of the Cambridge Philosophical Society, 26:528–535.
  13. Fisher, R. A. (1935). The fiducial argument in statistical inference. Ann. Eugenics, 6:391–398.
  14. Fraser, D. A. S. (2011). Is Bayes posterior just quick and dirty confidence? Statist. Sci., 26(3):299–316.
  15. Fraser, D. A. S. (2014). Why does statistics have two theories? In Lin, X., Genest, C., Banks, D. L., Molenberghs, G., Scott, D. W., and Wang, J.-L., editors, Past, Present, and Future of Statistical Science, chapter 22. Chapman & Hall/CRC Press.
  16. Hacking, I. (1976). Logic of Statistical Inference. Cambridge University Press, Cambridge-New York-Melbourne.
  17. Generalized fiducial inference: a review and new results. J. Amer. Statist. Assoc., 111(515):1346–1361.
  18. Hose, D. (2022). Possibilistic Reasoning with Imprecise Probabilities: Statistical Inference and Dynamic Filtering. PhD thesis, University of Stuttgart.
  19. Jeffreys, H. (1946). An invariant form for the prior probability in estimation problems. Proc. Roy. Soc. London Ser. A, 186:453–461.
  20. Martin, R. (2015). Plausibility functions and exact frequentist inference. J. Amer. Statist. Assoc., 110(512):1552–1561.
  21. Martin, R. (2018). On an inferential model construction using generalized associations. J. Statist. Plann. Inference, 195:105–115.
  22. Martin, R. (2019). False confidence, non-additive beliefs, and valid statistical inference. Internat. J. Approx. Reason., 113:39–73.
  23. Martin, R. (2021). An imprecise-probabilistic characterization of frequentist statistical inference. arXiv:2112.10904.
  24. Martin, R. (2022a). Valid and efficient imprecise-probabilistic inference with partial priors, I. First results. arXiv:2203.06703.
  25. Martin, R. (2022b). Valid and efficient imprecise-probabilistic inference with partial priors, II. General framework. arXiv:2211.14567.
  26. Martin, R. (2023a). Fiducial inference viewed through a possibility-theoretic inferential model lens. In Miranda, E., Montes, I., Quaeghebeur, E., and Vantaggi, B., editors, Proceedings of the Thirteenth International Symposium on Imprecise Probability: Theories and Applications, volume 215 of Proceedings of Machine Learning Research, pages 299–310. PMLR.
  27. Martin, R. (2023b). Fisher’s underworld and the behavioral–statistical reliability balance in scientific inference. arXiv:2312.14912.
  28. Martin, R. (2023c). A possibility-theoretic solution to Basu’s Bayesian–frequentist via media. Sankhya A, to appear, arXiv:2303.17425.
  29. Response to the comment ‘Confidence in confidence distributions!’. Proc. R. Soc. A., 477:20200579.
  30. Inferential Models, volume 147 of Monographs on Statistics and Applied Probability. CRC Press, Boca Raton, FL.
  31. Shafer, G. (1976). A Mathematical Theory of Evidence. Princeton University Press, Princeton, N.J.
  32. Shafer, G. (1982). Belief functions and parametric models. J. Roy. Statist. Soc. Ser. B, 44(3):322–352. With discussion.
  33. Stein, C. (1959). An example of wide discrepancy between fiducial and confidence intervals. Ann. Math. Statist., 30:877–880.
  34. van der Vaart, A. W. (1998). Asymptotic Statistics. Cambridge University Press, Cambridge.
  35. Walley, P. (1991). Statistical Reasoning with Imprecise Probabilities, volume 42 of Monographs on Statistics and Applied Probability. Chapman & Hall Ltd., London.
  36. Wasserman, L. A. (1990). Belief functions and statistical inference. Canad. J. Statist., 18(3):183–196.
  37. Confidence distribution, the frequentist distribution estimator of a parameter: a review. Int. Stat. Rev., 81(1):3–39.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets