Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
162 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Combining Evidence Across Filtrations Using Adjusters (2402.09698v2)

Published 15 Feb 2024 in stat.ME, cs.LG, math.PR, math.ST, stat.ML, and stat.TH

Abstract: In anytime-valid sequential inference, it is known that any admissible procedure must be based on e-processes, which are composite generalizations of test martingales that quantify the accumulated evidence against a composite null hypothesis at any arbitrary stopping time. This paper studies methods for combining e-processes constructed using different information sets (filtrations) for the same null. Although e-processes constructed in the same filtration can be combined effortlessly (e.g., by averaging), e-processes constructed in different filtrations cannot, because their validity in a coarser filtration does not translate to validity in a finer filtration. This issue arises in exchangeability tests, independence tests, and tests for comparing forecasts with lags. We first establish that a class of functions called adjusters allows us to lift e-processes from a coarser filtration into any finer filtration. We then introduce a characterization theorem for adjusters, formalizing a sense in which using adjusters is necessary. There are two major implications. First, if we have a powerful e-process in a coarsened filtration, then we readily have a powerful e-process in the original filtration. Second, when we coarsen the filtration to construct an e-process, there is an asymptotically logarithmic cost of recovering anytime-validity in the original filtration.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (56)
  1. Sequentially valid tests for forecast calibration. The Annals of Applied Statistics, 17(3):1909–1935.
  2. Sequential nonparametric testing with the law of the iterated logarithm. In Proceedings of the Thirty-Second Conference on Uncertainty in Artificial Intelligence, UAI’16, page 42–51.
  3. On certain probabilities equivalent to coin-tossing, d’après schachermayer. In Séminaire de Probabilités XXXIII, pages 240–256. Springer.
  4. Changes of filtrations and of probability measures. Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete, 45(4):269–295.
  5. Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78(1):1–3.
  6. Multiple testing under negative dependence. arXiv preprint arXiv:2212.09706.
  7. Comparing sequential forecasters. Operations Research.
  8. Cox, D. R. (1952). Sequential tests for composite hypotheses. Mathematical Proceedings of the Cambridge Philosophical Society, 48(2):290–299.
  9. Confidence sequences for mean, variance, and median. Proceedings of the National Academy of Sciences, 58(1):66–68.
  10. Probability-free pricing of adjusted American lookbacks. arXiv preprint arXiv:1108.4113.
  11. Insuring against loss of evidence in game-theoretic probability. Statistics & Probability Letters, 81(1):157–162.
  12. de Finetti, B. (1931). Funzione caratteristica di un fenomeno aleatorio. In Atti della R. Accademia Nazionale dei Lincei, Ser. 6, Memorie, Classe di Scienze Fisiche, Matematiche e Naturale, volume 4, pages 251–299.
  13. Decreasing sequences of σ𝜎\sigmaitalic_σ-fields and a measure change for Brownian motion. The Annals of Probability, 24(2):882–904.
  14. Durrett, R. (2019). Probability: Theory and examples, volume 49. Cambridge University Press.
  15. Plug-in martingales for testing exchangeability on-line. In Proceedings of International Conference on International Conference on Machine Learning, pages 923–930.
  16. Conformal martingales. Inventiones Mathematicae, 16(4):271–308.
  17. Safe testing. Journal of the Royal Statistical Society: Series B (Statistical Methodology).
  18. A rank-based sequential test of independence. arXiv preprint arXiv:2305.13818.
  19. Valid sequential inference on probability forecast performance. Biometrika, 109(3):647–663.
  20. Time-uniform, nonparametric, nonasymptotic confidence sequences. The Annals of Statistics, 49(2):1055 – 1080.
  21. E-values as unnormalized weights in multiple testing. Biometrika.
  22. Always valid inference: Continuous monitoring of A/B tests. Operations Research, 70(3):1806–1821.
  23. Lai, T. L. (1976). On confidence sequences. The Annals of Statistics, 4(2):265–280.
  24. E-statistics, group invariance and anytime valid testing. arXiv preprint arXiv:2208.07610.
  25. Sequential kernelized independence testing. International Conference on Machine Learning (ICML).
  26. Sequential predictive two-sample and independence testing. Advances in Neural Information Processing Systems (NeurIPS).
  27. Game-theoretic statistics and safe anytime-valid inference. Statistical Science.
  28. Randomized and exchangeable improvements of Markov’s, Chebyshev’s and Chernoff’s inequalities. arXiv preprint arXiv:2304.02611.
  29. Admissible anytime-valid sequential inference must rely on nonnegative martingales. arXiv preprint arXiv:2009.03167.
  30. Testing exchangeability: Fork-convexity, supermartingales and e-processes. International Journal of Approximate Reasoning, 141:83–109.
  31. Derandomised knockoffs: leveraging e-values for false discovery rate control. Journal of the Royal Statistical Society: Series B (Statistical Methodology).
  32. Robbins, H. (1970). Statistical methods related to the law of the iterated logarithm. The Annals of Mathematical Statistics, 41(5):1397–1409.
  33. Testing exchangeability by pairwise betting. International Conference on Artificial Intelligence and Statistics (AISTATS).
  34. Shafer, G. (2021). Testing by betting: A strategy for statistical and scientific communication. Journal of the Royal Statistical Society: Series A (Statistics in Society), 184(2):407–431.
  35. Test martingales, Bayes factors and p-values. Statistical Science, 26(1):84–101.
  36. Probability and finance: It’s only a game!, volume 491. Wiley.
  37. Game-theoretic foundations for probability and finance, volume 455. Wiley.
  38. Nonparametric two-sample testing by betting. IEEE Transactions on Information Theory.
  39. ALL-IN meta-analysis: Breathing life into living systematic reviews. F1000Research, 11.
  40. Tsirelson, B. (1998). Within and beyond the reach of Brownian innovation. In Proceedings of the International Congress of Mathematicians, volume 3, pages 311–320.
  41. Ville, J. (1939). Étude critique de la notion de collectif. Gauthier-Villars.
  42. Vovk, V. (2021a). Conformal testing in a binary model situation. In Conformal and Probabilistic Prediction and Applications, pages 131–150. PMLR.
  43. Vovk, V. (2021b). Testing randomness online. Statistical Science, 36(4):595–611.
  44. Algorithmic learning in a random world, volume 29. Springer.
  45. Testing exchangeability on-line. In Proceedings of the International Conference on Machine Learning, pages 768–775.
  46. Retrain or not retrain: Conformal test martingales for change-point detection. In Conformal and Probabilistic Prediction and Applications, pages 191–210. PMLR.
  47. Admissible ways of merging p-values under arbitrary dependence. The Annals of Statistics, 50(1):351–375.
  48. Combining p-values via averaging. Biometrika, 107(4):791–808.
  49. Merging sequential e-values via martingales. arXiv preprint arXiv:2007.06382.
  50. E-values: Calibration, combination and applications. The Annals of Statistics, 49(3):1736–1754.
  51. The extended Ville’s inequality for nonintegrable nonnegative supermartingales. arXiv preprint arXiv:2304.01163.
  52. False discovery rate control with e-values. Journal of the Royal Statistical Society Series B (Statistical Methodology), 84(3):822–852.
  53. Universal inference. Proceedings of the National Academy of Sciences, 117(29):16880–16890.
  54. Estimating means of bounded random variables by betting. Journal of the Royal Statistical Society: Series B (Statistical Methodology).
  55. A unified framework for bandit multiple testing. Advances in Neural Information Processing Systems, 34:16833–16845.
  56. Post-selection inference for e-value based confidence intervals. arXiv preprint arXiv:2203.12572.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets