PROTEST: Nonparametric Testing of Hypotheses Enhanced by Experts' Utility Judgements (2403.05655v1)
Abstract: Instead of testing solely a precise hypothesis, it is often useful to enlarge it with alternatives that are deemed to differ from it negligibly. For instance, in a bioequivalence study one might consider the hypothesis that the concentration of an ingredient is exactly the same in two drugs. In such a context, it might be more relevant to test the enlarged hypothesis that the difference in concentration between the drugs is of no practical significance. While this concept is not alien to Bayesian statistics, applications remain confined to parametric settings and strategies on how to effectively harness experts' intuitions are often scarce or nonexistent. To resolve both issues, we introduce PROTEST, an accessible nonparametric testing framework that seamlessly integrates with Markov Chain Monte Carlo (MCMC) methods. We develop expanded versions of the model adherence, goodness-of-fit, quantile and two-sample tests. To demonstrate how PROTEST operates, we make use of examples, simulated studies - such as testing link functions in a binary regression setting, as well as a comparison between the performance of PROTEST and the PTtest (Holmes et al., 2015) - and an application with data on neuron spikes. Furthermore, we address the crucial issue of selecting the threshold - which controls how much a hypothesis is to be expanded - even when intuitions are limited or challenging to quantify.
- Berger, J. O. (1985). Statistical decision theory and Bayesian analysis. New York: Springer-Verlag.
- “The Gender Wage Gap: Extent, Trends, and Explanations.” Journal of Economic Literature, 55(3): 789–865.
- Brezis, H. (2011). Functional Analysis, Sobolev Spaces and Partial Differential Equations. Springer New York.
- Statistical Inference. Duxbury Resource Center.
- “Comparing two populations using Bayesian Fourier series density estimation.” Communications in Statistics - Simulation and Computation, 49(1): 261–282.
- “WIKS: a general Bayesian nonparametric index for quantifying differences between two populations.” TEST, 30: 274–291.
- “A Fully Nonparametric Modeling Approach to Binary Regression.” Bayesian Analysis, 10(4): 821 – 847. URL https://doi.org/10.1214/15-BA963SI
- Duguid, H. A. (1969). “A study of the evaporation rates of small freely falling water droplets.” Master’s thesis, Missouri University of Science and Technology, Rolla, Missouri, USA.
- “Bayesian statistical inference for psychological research.” Psychological review, 70(3): 193.
- “The Logical Consistency of Simultaneous Agnostic Hypothesis Tests.” Entropy, 18(7).
- — (2019). “Pragmatic Hypotheses in the Evolution of Science.” Entropy, 21(9).
- — (2023). “Logical coherence in Bayesian simultaneous three-way hypothesis tests.” International Journal of Approximate Reasoning, 152: 297–309.
- “Dataset of human medial temporal lobe single neuron activity during declarative memory encoding and recognition.” Scientific Data, 5(1).
- Ferguson, T. S. (1973). “A Bayesian Analysis of Some Nonparametric Problems.” The Annals of Statistics, 1(2): 209 – 230.
- Statistical Tables for Biological, Agricultural and Medical Research, Sixth Edition. Hafner Publishing Company, 6th revised edition edition.
- Folland, G. (2013). Real Analysis: Modern Techniques and Their Applications. Pure and Applied Mathematics: A Wiley Series of Texts, Monographs and Tracts. Wiley.
- Spiking Neuron Models. Cambridge University Press.
- Goldin, C. (1990). Understanding the Gender Gap: An Economic History of American Women. American studies collection. Oxford University Press.
- Good, I. J. (2009). Some Logic and History of Hypothesis Testing, 129–148. Dover Books on Mathematics. Dover Publications.
- Gross, J. H. (2014). “Testing What Matters (If You Must Test at All): A Context-Driven Approach to Substantive and Statistical Significance.” American Journal of Political Science, 59(3): 775–788.
- “Modeling Regression Error With a Mixture of Polya Trees.” Journal of the American Statistical Association, 97(460): 1020–1033.
- “Practical Bayesian Design and Analysis for Drug and Device Clinical Trials.” Journal of Biopharmaceutical Statistics, 18(1): 54–80.
- “Testing the Approximate Validity of Statistical Hypotheses.” Journal of the Royal Statistical Society. Series B (Methodological), 16(2): 261–268.
- “Two-sample Bayesian Nonparametric Hypothesis Testing.” Bayesian Analysis, 10(2): 297 – 320.
- Jeffreys, H. (1961). Theory of Probability. Oxford, England: Oxford, third edition.
- Kass, R. E. (1993). “Bayes factors in practice.” Journal of the Royal Statistical Society. Series D (The Statistician), 42(5): 551–560.
- Kolmogorov, A. N. (1992). 15. On The Empirical Determination of A Distribution Law, 139–146. Dordrecht: Springer Netherlands.
- Kreyszig, E. (1978). Introductory Functional Analysis with Applications. Wiley classics library. Wiley.
- Kruschke, J. K. (2018). “Rejecting or Accepting Parameter Values in Bayesian Estimation.” Advances in Methods and Practices in Psychological Science, 1(2): 270–280.
- Lakens, D. (2022). “Improving Your Statistical Inferences.” URL https://zenodo.org/record/6409077
- “Equivalence Testing for Psychological Research: A Tutorial.” Advances in Methods and Practices in Psychological Science, 1(2): 259–269.
- Lavine, M. (1992). “Some Aspects of Polya Tree Distributions for Statistical Modelling.” The Annals of Statistics, 20(3): 1222–1235.
- — (1994). “More Aspects of Polya Tree Distributions for Statistical Modelling.” The Annals of Statistics, 22(3): 1161–1176.
- Leamer, E. E. (1988). “3 Things That Bother Me.” Economic Record, 64(4): 331–335.
- Statistical Inference: An Integrated Approach, Second Edition. Chapman & Hall/CRC Texts in Statistical Science. CRC Press.
- Mitchell, P. (2018). “Teaching statistical appreciation in quantitative methods.” MSOR Connections, 16(2): 37.
- “On the Problem of the Most Efficient Tests of Statistical Hypotheses.” Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character, 231: 289–337.
- Pett, M. A. (2016). Nonparametric Statistics for Health Care Research: Statistics for Small Samples and Unusual Distributions. SAGE Publications, Inc.
- R Core Team (2022). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/
- Monte Carlo statistical methods. Springer texts in statistics. Berlin: Springer, 2 edition.
- Ross, S. M. (2009). A First Course in Probability. Pearson, 8 edition.
- Schervish, M. (2012). Theory of Statistics. Springer Series in Statistics. Springer New York.
- Schervish, M. J. (1996). “P Values: What They Are and What They Are Not.” The American Statistician, 50(3): 203–206.
- “Logically-consistent hypothesis testing and the hexagon of oppositions.” Logic Journal of the IGPL, 25(5): 741–757.
- UNECE (2023). “UNECE Statistical Database.” https://w3.unece.org/PXWeb2015/pxweb/en/STAT/. Accessed: 2023-09-23.
- “Derivative Estimation Based on Difference Sequence via Locally Weighted Least Squares Regression.” Journal of Machine Learning Research, 16(81): 2617–2641.
- “Gaussian Processes for Regression.” In Advances in neural information processing systems 8, 514–520. Max-Planck-Gesellschaft, Cambridge, MA, USA: MIT Press.
Sponsor
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.