Papers
Topics
Authors
Recent
Search
2000 character limit reached

Evaluating Treatment Prioritization Rules via Rank-Weighted Average Treatment Effects

Published 15 Nov 2021 in stat.ME and stat.ML | (2111.07966v2)

Abstract: There are a number of available methods for selecting whom to prioritize for treatment, including ones based on treatment effect estimation, risk scoring, and hand-crafted rules. We propose rank-weighted average treatment effect (RATE) metrics as a simple and general family of metrics for comparing and testing the quality of treatment prioritization rules. RATE metrics are agnostic as to how the prioritization rules were derived, and only assess how well they identify individuals that benefit the most from treatment. We define a family of RATE estimators and prove a central limit theorem that enables asymptotically exact inference in a wide variety of randomized and observational study settings. RATE metrics subsume a number of existing metrics, including the Qini coefficient, and our analysis directly yields inference methods for these metrics. We showcase RATE in the context of a number of applications, including optimal targeting of aspirin to stroke patients.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (75)
  1. Cox’s regression model for counting processes: a large sample study. The Annals of Statistics, pages 1100–1120, 1982.
  2. Framingham risk score for prediction of cardiovascular diseases: a population-based study from southern Europe. PloS One, 8(9):e73529, 2013.
  3. E. Ascarza. Retention futility: Targeting high-risk customers might be ineffective. Journal of Marketing Research, 55(1):80–98, 2018.
  4. S. Athey and G. W. Imbens. Machine learning methods that economists should know about. Annual Review of Economics, 11:685–725, 2019.
  5. S. Athey and S. Wager. Policy learning with observational data. Econometrica, 89(1):133–161, 2021.
  6. Generalized Random Forests. Annals of Statistics, 47(2):1148–1178, 2019.
  7. Detecting heterogeneous treatment effects to guide personalized blood pressure treatment: a modeling study of randomized clinical trials. Annals of Internal Medicine, 166(5):354–360, 2017.
  8. Effects of intensive systolic blood pressure lowering on cardiovascular events and mortality in patients with type 2 diabetes mellitus on standard glycemic control and in those without diabetes mellitus: reconciling results from ACCORD BP and SPRINT. Journal of the American Heart Association, 7(18):e009326, 2018.
  9. L. Breiman. Random forests. Machine learning, 45(1):5–32, 2001.
  10. Z.-M. Chen et al. CAST: randomised placebo-controlled trial of early aspirin use in 20 000 patients with acute ischaemic stroke. The Lancet, 349(9066):1641–1649, 1997.
  11. Indications for early aspirin use in acute ischemic stroke: a combined analysis of 40 000 randomized patients from the Chinese Acute Stroke Trial and the International Stroke Trial. Stroke, 31(6):1240–1249, 2000.
  12. Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal, 21(1):C1–C68, 2018a.
  13. Generic machine learning inference on heterogenous treatment effects in randomized experiments. Technical report, National Bureau of Economic Research, 2018b.
  14. Locally robust semiparametric estimation. Econometrica, 90(4):1501–1535, 2022.
  15. E. Chung and J. P. Romano. Exact and asymptotically robust permutation tests. The Annals of Statistics, 41(2):484–507, 2013.
  16. Estimating heterogeneous treatment effects with right-censored data via causal survival forests. Journal of the Royal Statistical Society Series B: Statistical Methodology, 85(2):179–211, 2023.
  17. General Cardiovascular Risk Profile for Use in Primary Care. Circulation, 117(6):743–753, 2008.
  18. An analysis of calibration and discrimination among multiple cardiovascular risk scores in a modern multiethnic cohort. Annals of Internal Medicine, 162(4):266–275, 2015.
  19. A Large Scale Benchmark for Uplift Modeling. In KDD, 2018.
  20. Randomization inference for treatment effect variation. Journal of the Royal Statistical Society: Series B: Statistical Methodology, pages 655–671, 2016.
  21. Asymptotic Minimax Character of the Sample Distribution Function and of the Classical Multinomial Estimator. The Annals of Mathematical Statistics, 27(3):642 – 669, 1956. doi: 10.1214/aoms/1177728174. URL https://doi.org/10.1214/aoms/1177728174.
  22. B. Efron. The jackknife, the bootstrap and other resampling plans. SIAM, 1982.
  23. Global, regional, and national burden of stroke and its risk factors, 1990–2019: A systematic analysis for the Global Burden of Disease Study 2019. The Lancet Neurology, 20(10):795–820, 2021.
  24. N. D. Fisher and G. Curfman. Hypertension—a public health challenge of global proportions. Journal of the American Medical Association, 320(17):1757–1759, 2018.
  25. J. H. Friedman. Multivariate Adaptive Regression Splines. The Annals of Statistics, pages 1–67, 1991.
  26. 2013 ACC/AHA Guideline on the Assessment of Cardiovascular Risk. Circulation, 129(25_suppl_2):S49–S73, 2014. doi: 10.1161/01.cir.0000437741.48606.98. URL https://www.ahajournals.org/doi/abs/10.1161/01.cir.0000437741.48606.98.
  27. A comparison of approaches to advertising measurement: Evidence from big field experiments at Facebook. Marketing Science, 38(2):193–225, 2019.
  28. Signal Detection Theory and Psychophysics, volume 1. Wiley New York, 1966.
  29. A. S. Group. Effects of intensive blood-pressure control in type 2 diabetes mellitus. New England Journal of Medicine, 362(17):1575–1585, 2010.
  30. I. S. T. C. Group. The International Stroke Trial (IST): a randomised trial of aspirin, subcutaneous heparin, both, or neither among 19 435 patients with acute ischaemic stroke. The Lancet, 349(9065):1569–1581, 1997.
  31. S. R. Group. A randomized trial of intensive versus standard blood-pressure control. New England Journal of Medicine, 373(22):2103–2116, 2015.
  32. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer Science & Business Media, 2009.
  33. J. L. Hill. Bayesian nonparametric modeling for causal inference. Journal of Computational and Graphical Statistics, 20(1):217–240, 2011.
  34. Focus on the Long-Term: It’s better for Users and Business. In Proceedings 21st Conference on Knowledge Discovery and Data Mining, Sydney, Australia, 2015. URL http://dl.acm.org/citation.cfm?doid=2783258.2788583.
  35. K. Imai and M. L. Li. Experimental evaluation of individualized treatment rules. Journal of the American Statistical Association, 118(541):242–256, 2023.
  36. Causal inference in statistics, social, and biomedical sciences. Cambridge University Press, 2015.
  37. Random survival forests. Annals of Applied Statistics, 2(3):841–860, 2008.
  38. Ghost Ads: Improving the Economics of Measuring Online Ad Effectiveness. Journal of Marketing Research, 54(6):867–884, 2017.
  39. S. Kaul. A tale of two trials: reconciling differences in results by exploring heterogeneous treatment effects. Annals of Internal Medicine, 166(5):370–372, 2017.
  40. E. H. Kennedy. Towards optimal doubly robust estimation of heterogeneous causal effects. Electronic Journal of Statistics, 17(2):3008–3049, 2023.
  41. Risk and treatment effect heterogeneity: re-analysis of individual participant data from 32 large clinical trials. International journal of epidemiology, 45(6):2075–2088, 2016.
  42. The Predictive Approaches to Treatment effect Heterogeneity (PATH) Statement. Annals of Internal Medicine, 172(1):35–45, 2020.
  43. Metalearners for estimating heterogeneous treatment effects using machine learning. Proceedings of the National Academy of Sciences, 116(10):4156–4165, 2019.
  44. E. Mammen. When does bootstrap work?: Asymptotic results and simulations, volume 77. Springer Science & Business Media, 2012.
  45. C. F. Manski. Statistical treatment rules for heterogeneous populations. Econometrica, 72(4):1221–1246, 2004.
  46. Potential US population impact of the 2017 ACC/AHA high blood pressure guideline. Circulation, 137(2):109–118, 2018.
  47. X. Nie and S. Wager. Quasi-oracle estimation of heterogeneous treatment effects. Biometrika, 108(2):299–319, 2021.
  48. Individualising intensive systolic blood pressure reduction in hypertension using computational trial phenomaps and machine learning: a post-hoc analysis of randomised clinical trials. The Lancet Digital Health, 4(11):e796–e805, 2022.
  49. Primary stroke prevention worldwide: translating evidence into action. The Lancet Public Health, 2021.
  50. Guidelines for the early management of patients with acute ischemic stroke: 2019 update to the 2018 guidelines for the early management of acute ischemic stroke: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke, 50(12):e344–e418, 2019.
  51. J. Præstgaard and J. A. Wellner. Exchangeably weighted bootstraps of the general empirical process. The Annals of Probability, pages 2053–2086, 1993.
  52. Health outcomes associated with various antihypertensive therapies used as first-line agents: a network meta-analysis. Journal of the American Medical Association, 289(19):2534–2544, 2003.
  53. N. Radcliffe and P. Surry. Differential response analysis: Modeling true responses by isolating the effect of a single action. Credit Scoring and Credit Control IV, 1999.
  54. N. J. Radcliffe. Using control groups to target on predicted lift: Building and assessing uplift models. Direct Marketing Analytics Journal, 1(3):14–21, 2007.
  55. Predictive approaches to heterogeneous treatment effects: a scoping review. BMC Medical Research Methodology, 20(1):1–12, 2020.
  56. Estimation of regression coefficients when some regressors are not always observed. Journal of the American statistical Association, 89(427):846–866, 1994.
  57. The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1):41–55, 1983.
  58. The international stroke trial database. Trials, 12(1):1–7, 2011.
  59. Stroke outcome in those over 80: a multicenter cohort study across Canada. Stroke, 39(8):2310–2317, 2008.
  60. V. Semenova and V. Chernozhukov. Debiased machine learning of conditional average treatment effects and other causal functions. The Econometrics Journal, 24(2):264–289, 2021.
  61. G. R. Shorack. Functions of order statistics. The Annals of Mathematical Statistics, 43(2):412–427, 1972.
  62. Empirical Processes with Applications to Statistics. SIAM, 2009.
  63. Treatment Allocation under Uncertain Costs. arXiv preprint arXiv:2103.11066, 2021.
  64. The Cox model. In Modeling Survival Data: Extending the Cox Model, pages 39–77. Springer, 2000.
  65. A. Tsiatis. Semiparametric theory and missing data. Springer Science & Business Media, 2007.
  66. Changes in functional outcome over the first year after stroke: an observational study from the Swedish stroke register. Stroke, 46(2):389–394, 2015.
  67. A. W. Van Der Vaart and J. A. Wellner. Weak Convergence. In Weak Convergence and Empirical Processes, pages 16–28. Springer, 1996.
  68. S. Wager and S. Athey. Estimation and inference of heterogeneous treatment effects using random forests. Journal of the American Statistical Association, 113(523):1228–1242, 2018.
  69. 2017 ACC/AHA/AAPA/ABC/ACPM/AGS/APhA/ASH/ASPC/NMA/PCNA guideline for the prevention, detection, evaluation, and management of high blood pressure in adults: a report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines. Journal of the American College of Cardiology, 71(19):e127–e248, 2018.
  70. Treatment heterogeneity with survival outcomes. In Handbook of Matching and Weighting Adjustments for Causal Inference, pages 445–482. Chapman and Hall/CRC, 2023.
  71. Clinical implications of revised pooled cohort equations for estimating atherosclerotic cardiovascular disease risk. Annals of Internal Medicine, 169(1):20–29, 2018.
  72. Effectively selecting a target population for a future comparative study. Journal of the American Statistical Association, 108(502):527–539, 2013.
  73. Cross-validated targeted minimum-loss-based estimation. In Targeted Learning, pages 459–474. Springer, 2011.
  74. M. H. Zweig and G. Campbell. Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. Clinical Chemistry, 39(4):561–577, 1993.
  75. A. W. van der Vaart. Asymptotic Statistics. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, 1998.
Citations (53)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up