Papers
Topics
Authors
Recent
Detailed Answer
Quick Answer
Concise responses based on abstracts only
Detailed Answer
Well-researched responses based on abstracts and relevant paper content.
Custom Instructions Pro
Preferences or requirements that you'd like Emergent Mind to consider when generating responses
Gemini 2.5 Flash
Gemini 2.5 Flash 63 tok/s
Gemini 2.5 Pro 49 tok/s Pro
GPT-5 Medium 14 tok/s Pro
GPT-5 High 19 tok/s Pro
GPT-4o 100 tok/s Pro
Kimi K2 174 tok/s Pro
GPT OSS 120B 472 tok/s Pro
Claude Sonnet 4 37 tok/s Pro
2000 character limit reached

Asymptotic Validity and Finite-Sample Properties of Approximate Randomization Tests (1908.04218v3)

Published 12 Aug 2019 in stat.ME and stat.ML

Abstract: Randomization tests rely on simple data transformations and possess an appealing robustness property. In addition to being finite-sample valid if the data distribution is invariant under the transformation, these tests can be asymptotically valid under a suitable studentization of the test statistic, even if the invariance does not hold. However, practical implementation often encounters noisy data, resulting in approximate randomization tests that may not be as robust. In this paper, our key theoretical contribution is a non-asymptotic bound on the discrepancy between the size of an approximate randomization test and the size of the original randomization test using noiseless data. This allows us to derive novel conditions for the validity of approximate randomization tests under data invariances, while being able to leverage existing results based on studentization if the invariance does not hold. We illustrate our theory through several examples, including tests of significance in linear regression. Our theory can explain certain aspects of how randomization tests perform in small samples, addressing limitations of prior theoretical results.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (41)
  1. Some asymptotic theory for the bootstrap. The annals of statistics, 9(6):1196–1217, 1981.
  2. Randomization tests under an approximate symmetry assumption. Econometrica, 85(3):1013–1030, 2017.
  3. An exact and robust conformal inference method for counterfactual and synthetic controls. Journal of the American Statistical Association, 116(536):1849–1864, 2021.
  4. Exact and asymptotically robust permutation tests. The Annals of Statistics, 41(2):484–507, 2013.
  5. Theoretical statistics. Chapman and Hall/CRC, 1979.
  6. The wild bootstrap, tamed at last. Journal of Econometrics, 146(1):162–169, 2008.
  7. Robust permutation tests for correlation and regression coefficients. Journal of the American Statistical Association, 112(519):1211–1220, 2017.
  8. Edgar Dobriban. Consistency of invariance-based randomization tests. arXiv preprint arXiv:2104.12260, 2021.
  9. Morris L Eaton. Group invariance applications in statistics. IMS, 1989.
  10. Randomization tests. Chapman and Hall/CRC, 2007.
  11. Bradley Efron. Student’s t-test under symmetry conditions. Journal of the American Statistical Association, 64(328):1278–1302, 1969.
  12. Modelling extremal events: for insurance and finance, volume 33. Springer Science & Business Media, 2013.
  13. Michael D Ernst. Permutation methods: a basis for exact inference. Statistical Science, pages 676–685, 2004.
  14. Ronald Aylmer Fisher. The design of experiments. Oliver And Boyd; Edinburgh, 1935.
  15. Colin B Fogarty. Testing weak nulls in matched observational studies. Biometrics, 79(3):2196–2207, 2023.
  16. A nonstochastic interpretation of reported significance levels. Journal of Business & Economic Statistics, 1(4):292–298, 1983a.
  17. David A Freedman. Bootstrapping regression models. The Annals of Statistics, 9(6):1218–1228, 1981.
  18. Significance testing in a nonstochastic setting. A festschrift for Erich L. Lehmann, pages 185–208, 1983b.
  19. Field experiments: Design, analysis, and interpretation. WW Norton, 2012.
  20. Phillip Good. Permutation tests: a practical guide to resampling methods for testing hypotheses. Springer Science & Business Media, 2013.
  21. James J Higgins. An introduction to modern nonparametric statistics. Brooks/Cole Pacific Grove, CA, 2004.
  22. Design and analysis of experiments, volume 1: Introduction to experimental design, volume 1. John Wiley & Sons, 2007.
  23. Wassily Hoeffding. The Large-Sample Power of Tests Based on Permutations of Observations. The Annals of Mathematical Statistics, 23(2):169 – 192, 1952. doi: 10.1214/aoms/1177729436. URL https://doi.org/10.1214/aoms/1177729436.
  24. Peter J Huber. Robust regression: asymptotics, conjectures and monte carlo. The annals of statistics, pages 799–821, 1973.
  25. Causal inference in statistics, social, and biomedical sciences. Cambridge University Press, 2015.
  26. Arnold Janssen. Studentized permutation tests for non-iid hypotheses and the generalized behrens-fisher problem. Statistics & probability letters, 36(1):9–21, 1997.
  27. Arnold Janssen. Nonparametric symmetry tests for statistical functionals. Mathematical Methods of Statistics, 8(3):320–343, 1999.
  28. Testing statistical hypotheses. New York: Springer, 2005.
  29. Regression adjustment in completely randomized experiments with a diverging number of covariates. Biometrika, 108(4):815–828, 2021.
  30. Regina Y Liu et al. Bootstrap procedures under some non-iid models. The Annals of Statistics, 16(4):1696–1708, 1988.
  31. Miles Lopes. A residual bootstrap for high-dimensional regression with near low-rank designs. Advances in Neural Information Processing Systems, 27, 2014.
  32. Enno Mammen et al. Bootstrap and wild bootstrap for high dimensional linear models. The annals of statistics, 21(1):255–285, 1993.
  33. Edwin JG Pitman. Significance tests which may be applied to samples from any populations. Supplement to the Journal of the Royal Statistical Society, 4(1):119–130, 1937.
  34. Joseph P Romano. Bootstrap and randomization tests of some nonparametric hypotheses. The Annals of Statistics, pages 141–159, 1989.
  35. Paul R Rosenbaum. Observational studies. In Observational studies, pages 1–17. Springer, 2002.
  36. Donald B Rubin. Randomization analysis of experimental data: The fisher randomization test comment. Journal of the American Statistical Association, 75(371):591–593, 1980.
  37. A. Tsybakov. Introduction to nonparametric estimation. Springer, 2009.
  38. Jeffrey M Wooldridge. Econometric analysis of cross section and panel data. MIT press, 2010.
  39. Chien-Fu Jeff Wu. Jackknife, bootstrap and other resampling methods in regression analysis. Annals of Statistics, 14(4):1261–1295, 1986.
  40. Randomization tests for weak null hypotheses in randomized experiments. Journal of the American Statistical Association, 116(536):1898–1913, 2021.
  41. Covariate-adjusted fisher randomization tests for the average treatment effect. Journal of Econometrics, 225(2):278–294, 2021.
Citations (2)
View on Semantic Scholar
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Follow-Up Questions

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Authors (1)

  1. Panos Toulis
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets