Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Who Are We Missing? A Principled Approach to Characterizing the Underrepresented Population (2401.14512v4)

Published 25 Jan 2024 in stat.ME, cs.CY, cs.LG, and stat.AP

Abstract: Randomized controlled trials (RCTs) serve as the cornerstone for understanding causal effects, yet extending inferences to target populations presents challenges due to effect heterogeneity and underrepresentation. Our paper addresses the critical issue of identifying and characterizing underrepresented subgroups in RCTs, proposing a novel framework for refining target populations to improve generalizability. We introduce an optimization-based approach, Rashomon Set of Optimal Trees (ROOT), to characterize underrepresented groups. ROOT optimizes the target subpopulation distribution by minimizing the variance of the target average treatment effect estimate, ensuring more precise treatment effect estimations. Notably, ROOT generates interpretable characteristics of the underrepresented population, aiding researchers in effective communication. Our approach demonstrates improved precision and interpretability compared to alternatives, as illustrated with synthetic data experiments. We apply our methodology to extend inferences from the Starting Treatment with Agonist Replacement Therapies (START) trial -- investigating the effectiveness of medication for opioid use disorder -- to the real-world population represented by the Treatment Episode Dataset: Admissions (TEDS-A). By refining target populations using ROOT, our framework offers a systematic approach to enhance decision-making accuracy and inform future trials in diverse populations.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (43)
  1. Abuse, S. et al. (2020). Treatment episode data set admissions (teds-a).
  2. Recovering from selection bias in causal and statistical inference. In Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, AAAI’14, pages 2410–2416. AAAI Press.
  3. Estimates of external validity bias when impact evaluations select sites nonrandomly. Educational Evaluation and Policy Analysis, 38(2):318–335.
  4. Advancing the inclusion of underrepresented women in clinical research. Cell Reports Medicine, 3(4).
  5. Toward national estimates of effectiveness of treatment for substance use. The Journal of clinical psychiatry, 78(1):7695.
  6. Boden-Albala, B. (2022). Confronting legacies of underrepresentation in clinical trials: The case for greater diversity in research. Neuron, 110(5):746–748.
  7. Breiman, L. (1996). Bagging predictors. Machine learning, 24:123–140.
  8. Breiman, L. (2001). Statistical modeling: The two cultures (with comments and a rejoinder by the author). Statistical science, 16(3):199–231.
  9. Breiman, L. (2017). Classification and regression trees. Routledge.
  10. How do we know when research from one setting can be useful in another? A review of external validity, applicability and transferability frameworks. Journal of health services research & policy, 16(4):238–244.
  11. Computer-aided assessment of the generalizability of clinical trial results. International journal of medical informatics, 99:60–66.
  12. Bart: Bayesian additive regression trees.
  13. Dealing with limited overlap in estimation of average treatment effects. Biometrika, 96(1):187–199.
  14. A review of generalizability and transportability. Annual Review of Statistics and Its Application, 10:501–524.
  15. How to assess the external validity of therapeutic trials: A conceptual approach. International Journal of Epidemiology, 39:89–94.
  16. Ding, P. (2023). A first course in causal inference. arXiv preprint arXiv:2305.18793.
  17. Randomization inference for treatment effect variation. Journal of the Royal Statistical Society. Series B, Statistical methodology, 78(3):655–671.
  18. Friedman, J. H. (1991). Multivariate adaptive regression splines. The annals of statistics, 19(1):1–67.
  19. Can we identify women at risk of pregnancy despite using emergency contraception? data from randomized trials of ulipristal acetate and levonorgestrel. Contraception, 84(4):363–367.
  20. Evaluating the relevance, generalization, and applicability of research: Issues in external validation and translation methodology. Evaluation & the health professions, 29:126–153.
  21. Generalizing from clinical trial data: A case study. the risk of suicidality among pediatric antidepressant users. Statistics in Medicine, 27(11):1801–13.
  22. Generalizing from clinical trial data: a case study. the risk of suicidality among pediatric antidepressant users. Statistics in medicine, 27(11):1801–1813.
  23. Treatment retention among patients randomized to buprenorphine/naloxone compared to methadone in a multi-site trial. Addiction, 109(1):79–87.
  24. Misunderstandings between experimentalists and observationalists about causal inference. Journal of the Royal Statistical Society: Series A (Statistics in Society), 171(2):481–502.
  25. Wide-minima density hypothesis and the explore-exploit learning rate schedule. Journal of Machine Learning Research, 24(65):1–37.
  26. Addressing extreme propensity scores via the overlap weights. American journal of epidemiology, 188(1):250–257.
  27. Generalized and scalable optimal sparse decision trees. In International Conference on Machine Learning, pages 6150–6160. PMLR.
  28. External validity in policy evaluations that choose sites purposively: External validity in policy evaluations. Journal of policy analysis and management, 32(1):107–121.
  29. Transportability of causal and statistical relations: A formal approach. In 2011 IEEE 11th International Conference on Data Mining Workshops, pages 540–547, Vancouver, BC, Canada. IEEE.
  30. External validity: From do-calculus to transportability across populations. Statistical Science, 29(4):579–595.
  31. Rothwell, P. M. (2005). External validity of randomised controlled trials: “to whom do the results of this trial apply?”. The Lancet, 365(9453):82–93.
  32. Robust estimation of encouragement design intervention effects transported across sites. Journal of the Royal Statistical Society Series B: Statistical Methodology, 79(5):1509–1525.
  33. Under-representation of key demographic groups in opioid use disorder trials. Drug and alcohol dependence reports, 4:100084.
  34. Efficiently transporting average treatment effects using a sufficient subset of effect modifiers. arXiv preprint arXiv:2304.00117.
  35. Buprenorphine/naloxone and methadone effects on laboratory indices of liver health: a randomized trial. Drug and alcohol dependence, 128(1-2):71–76.
  36. On the existence of simpler machine learning models. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 1827–1858.
  37. The use of propensity scores to assess the generalizability of results from randomized trials: Use of propensity scores to assess generalizability. Journal of the Royal Statistical Society: Series A (Statistics in Society), 174(2):369–386.
  38. Propensity score weighting and trimming strategies for reducing variance and bias of treatment effect estimates: a simulation study. American journal of epidemiology, 190(8):1659–1670.
  39. Assessing sample representativeness in randomized controlled trials: application to the national institute of drug abuse clinical trials network. Addiction, 111(7):1226–1234.
  40. The national drug abuse treatment clinical trials network: Forging a partnership between research knowledge and community practice. Substance Abuse and Rehabilitation, 2:21.
  41. Tipton, E. (2014). How generalizable is your experiment? An index for comparing experimental samples and populations. Journal of Educational and Behavioral Statistics, 39(6):478–501.
  42. A review of statistical methods for generalizing from evaluations of educational interventions. Educational Researcher, 47(8):516–524.
  43. A design-based approach to improve external validity in welfare policy evaluations. Evaluation Review, 41(4):326–356.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now