Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
149 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Combining Evidence Across Filtrations (2402.09698v3)

Published 15 Feb 2024 in stat.ME, cs.LG, math.PR, math.ST, stat.ML, and stat.TH

Abstract: In sequential anytime-valid inference, any admissible procedure must be based on e-processes: generalizations of test martingales that quantify the accumulated evidence against a composite null hypothesis at any stopping time. This paper proposes a method for combining e-processes constructed in different filtrations but for the same null. Although e-processes in the same filtration can be combined effortlessly (by averaging), e-processes in different filtrations cannot because their validity in a coarser filtration does not translate to a finer filtration. This issue arises in sequential tests of randomness and independence, as well as in the evaluation of sequential forecasters. We establish that a class of functions called adjusters can lift arbitrary e-processes across filtrations. The result yields a generally applicable "adjust-then-combine" procedure, which we demonstrate on the problem of testing randomness in real-world financial data. Furthermore, we prove a characterization theorem for adjusters that formalizes a sense in which using adjusters is necessary. There are two major implications. First, if we have a powerful e-process in a coarsened filtration, then we readily have a powerful e-process in the original filtration. Second, when we coarsen the filtration to construct an e-process, there is a logarithmic cost to recovering validity in the original filtration.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (56)
  1. Sequentially valid tests for forecast calibration. The Annals of Applied Statistics, 17(3):1909–1935.
  2. Sequential nonparametric testing with the law of the iterated logarithm. In Proceedings of the Thirty-Second Conference on Uncertainty in Artificial Intelligence, UAI’16, page 42–51.
  3. On certain probabilities equivalent to coin-tossing, d’après schachermayer. In Séminaire de Probabilités XXXIII, pages 240–256. Springer.
  4. Changes of filtrations and of probability measures. Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete, 45(4):269–295.
  5. Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78(1):1–3.
  6. Multiple testing under negative dependence. arXiv preprint arXiv:2212.09706.
  7. Comparing sequential forecasters. Operations Research.
  8. Cox, D. R. (1952). Sequential tests for composite hypotheses. Mathematical Proceedings of the Cambridge Philosophical Society, 48(2):290–299.
  9. Confidence sequences for mean, variance, and median. Proceedings of the National Academy of Sciences, 58(1):66–68.
  10. Probability-free pricing of adjusted American lookbacks. arXiv preprint arXiv:1108.4113.
  11. Insuring against loss of evidence in game-theoretic probability. Statistics & Probability Letters, 81(1):157–162.
  12. de Finetti, B. (1931). Funzione caratteristica di un fenomeno aleatorio. In Atti della R. Accademia Nazionale dei Lincei, Ser. 6, Memorie, Classe di Scienze Fisiche, Matematiche e Naturale, volume 4, pages 251–299.
  13. Decreasing sequences of σ𝜎\sigmaitalic_σ-fields and a measure change for Brownian motion. The Annals of Probability, 24(2):882–904.
  14. Durrett, R. (2019). Probability: Theory and examples, volume 49. Cambridge University Press.
  15. Plug-in martingales for testing exchangeability on-line. In Proceedings of International Conference on International Conference on Machine Learning, pages 923–930.
  16. Conformal martingales. Inventiones Mathematicae, 16(4):271–308.
  17. Safe testing. Journal of the Royal Statistical Society: Series B (Statistical Methodology).
  18. A rank-based sequential test of independence. arXiv preprint arXiv:2305.13818.
  19. Valid sequential inference on probability forecast performance. Biometrika, 109(3):647–663.
  20. Time-uniform, nonparametric, nonasymptotic confidence sequences. The Annals of Statistics, 49(2):1055 – 1080.
  21. E-values as unnormalized weights in multiple testing. Biometrika.
  22. Always valid inference: Continuous monitoring of A/B tests. Operations Research, 70(3):1806–1821.
  23. Lai, T. L. (1976). On confidence sequences. The Annals of Statistics, 4(2):265–280.
  24. E-statistics, group invariance and anytime valid testing. arXiv preprint arXiv:2208.07610.
  25. Sequential kernelized independence testing. International Conference on Machine Learning (ICML).
  26. Sequential predictive two-sample and independence testing. Advances in Neural Information Processing Systems (NeurIPS).
  27. Game-theoretic statistics and safe anytime-valid inference. Statistical Science.
  28. Randomized and exchangeable improvements of Markov’s, Chebyshev’s and Chernoff’s inequalities. arXiv preprint arXiv:2304.02611.
  29. Admissible anytime-valid sequential inference must rely on nonnegative martingales. arXiv preprint arXiv:2009.03167.
  30. Testing exchangeability: Fork-convexity, supermartingales and e-processes. International Journal of Approximate Reasoning, 141:83–109.
  31. Derandomised knockoffs: leveraging e-values for false discovery rate control. Journal of the Royal Statistical Society: Series B (Statistical Methodology).
  32. Robbins, H. (1970). Statistical methods related to the law of the iterated logarithm. The Annals of Mathematical Statistics, 41(5):1397–1409.
  33. Testing exchangeability by pairwise betting. International Conference on Artificial Intelligence and Statistics (AISTATS).
  34. Shafer, G. (2021). Testing by betting: A strategy for statistical and scientific communication. Journal of the Royal Statistical Society: Series A (Statistics in Society), 184(2):407–431.
  35. Test martingales, Bayes factors and p-values. Statistical Science, 26(1):84–101.
  36. Probability and finance: It’s only a game!, volume 491. Wiley.
  37. Game-theoretic foundations for probability and finance, volume 455. Wiley.
  38. Nonparametric two-sample testing by betting. IEEE Transactions on Information Theory.
  39. ALL-IN meta-analysis: Breathing life into living systematic reviews. F1000Research, 11.
  40. Tsirelson, B. (1998). Within and beyond the reach of Brownian innovation. In Proceedings of the International Congress of Mathematicians, volume 3, pages 311–320.
  41. Ville, J. (1939). Étude critique de la notion de collectif. Gauthier-Villars.
  42. Vovk, V. (2021a). Conformal testing in a binary model situation. In Conformal and Probabilistic Prediction and Applications, pages 131–150. PMLR.
  43. Vovk, V. (2021b). Testing randomness online. Statistical Science, 36(4):595–611.
  44. Algorithmic learning in a random world, volume 29. Springer.
  45. Testing exchangeability on-line. In Proceedings of the International Conference on Machine Learning, pages 768–775.
  46. Retrain or not retrain: Conformal test martingales for change-point detection. In Conformal and Probabilistic Prediction and Applications, pages 191–210. PMLR.
  47. Admissible ways of merging p-values under arbitrary dependence. The Annals of Statistics, 50(1):351–375.
  48. Combining p-values via averaging. Biometrika, 107(4):791–808.
  49. Merging sequential e-values via martingales. arXiv preprint arXiv:2007.06382.
  50. E-values: Calibration, combination and applications. The Annals of Statistics, 49(3):1736–1754.
  51. The extended Ville’s inequality for nonintegrable nonnegative supermartingales. arXiv preprint arXiv:2304.01163.
  52. False discovery rate control with e-values. Journal of the Royal Statistical Society Series B (Statistical Methodology), 84(3):822–852.
  53. Universal inference. Proceedings of the National Academy of Sciences, 117(29):16880–16890.
  54. Estimating means of bounded random variables by betting. Journal of the Royal Statistical Society: Series B (Statistical Methodology).
  55. A unified framework for bandit multiple testing. Advances in Neural Information Processing Systems, 34:16833–16845.
  56. Post-selection inference for e-value based confidence intervals. arXiv preprint arXiv:2203.12572.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets