Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Optimized Model Selection for Estimating Treatment Effects from Costly Simulations of the US Opioid Epidemic (2403.15755v1)

Published 23 Mar 2024 in stat.ME, cs.MA, cs.SI, and stat.AP

Abstract: Agent-based simulation with a synthetic population can help us compare different treatment conditions while keeping everything else constant within the same population (i.e., as digital twins). Such population-scale simulations require large computational power (i.e., CPU resources) to get accurate estimates for treatment effects. We can use meta models of the simulation results to circumvent the need to simulate every treatment condition. Selecting the best estimating model at a given sample size (number of simulation runs) is a crucial problem. Depending on the sample size, the ability of the method to estimate accurately can change significantly. In this paper, we discuss different methods to explore what model works best at a specific sample size. In addition to the empirical results, we provide a mathematical analysis of the MSE equation and how its components decide which model to select and why a specific method behaves that way in a range of sample sizes. The analysis showed why the direction estimation method is better than model-based methods in larger sample sizes and how the between-group variation and the within-group variation affect the MSE equation.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (23)
  1. J. C. Cajka, P. C. Cooley, and W. D. Wheaton, “Attribute assignment to a synthetic population in support of agent-based disease modeling,” Methods report (RTI Press), vol. 19, no. 1009, p. 1, 2010.
  2. N. M. Ferguson, D. A. Cummings, C. Fraser, J. C. Cajka, P. C. Cooley, and D. S. Burke, “Strategies for mitigating an influenza pandemic,” Nature, vol. 442, no. 7101, pp. 448–452, 2006.
  3. D. L. Chao, M. E. Halloran, V. J. Obenchain, and I. M. Longini Jr, “Flute, a publicly available stochastic influenza epidemic simulation model,” PLoS computational biology, vol. 6, no. 1, p. e1000656, 2010.
  4. J. Parker and J. M. Epstein, “A distributed platform for global-scale agent-based models of disease transmission,” ACM Transactions on Modeling and Computer Simulation (TOMACS), vol. 22, no. 1, pp. 1–25, 2011.
  5. A. R. Tuite, A. L. Greer, M. Whelan, A.-L. Winter, B. Lee, P. Yan, J. Wu, S. Moghadas, D. Buckeridge, B. Pourbohloul et al., “Estimated epidemiologic parameters and morbidity associated with pandemic h1n1 influenza,” Cmaj, vol. 182, no. 2, pp. 131–136, 2010.
  6. J. J. Grefenstette, S. T. Brown, R. Rosenfeld, J. DePasse, N. T. Stone, P. C. Cooley, W. D. Wheaton, A. Fyshe, D. D. Galloway, A. Sriram et al., “Fred (a framework for reconstructing epidemic dynamics): an open-source software system for modeling infectious diseases and control strategies using census-based populations,” BMC public health, vol. 13, no. 1, pp. 1–14, 2013.
  7. S. Lukens, J. DePasse, R. Rosenfeld, E. Ghedin, E. Mochan, S. T. Brown, J. Grefenstette, D. S. Burke, D. Swigon, and G. Clermont, “A large-scale immuno-epidemiological simulation of influenza a epidemics,” BMC public health, vol. 14, pp. 1–15, 2014.
  8. M. A. Potter, S. T. Brown, P. C. Cooley, P. M. Sweeney, T. B. Hershey, S. M. Gleason, B. Y. Lee, C. R. Keane, J. Grefenstette, and D. S. Burke, “School closure as an influenza mitigation strategy: how variations in legal authority and plan criteria can alter the impact,” BMC public health, vol. 12, pp. 1–11, 2012.
  9. F. Liu, W. T. Enanoria, J. Zipprich, S. Blumberg, K. Harriman, S. F. Ackley, W. D. Wheaton, J. L. Allpress, and T. C. Porco, “The role of vaccination coverage, individual behaviors, and the public health response in the control of measles epidemics: an agent-based simulation for california,” BMC public health, vol. 15, no. 1, pp. 1–16, 2015.
  10. M. G. Krauland, R. J. Frankeny, J. Lewis, L. Brink, E. G. Hulsey, M. S. Roberts, and K. A. Hacker, “Development of a synthetic population model for assessing excess risk for cardiovascular disease death,” JAMA network open, vol. 3, no. 9, pp. e2 015 047–e2 015 047, 2020.
  11. A. A. Ahmed, M. A. Rahimian, and M. S. Roberts, “Estimating treatment effects using costly simulation samples from a population-scale model of opioid use disorder,” in 2023 IEEE EMBS International Conference on Biomedical and Health Informatics (BHI).   IEEE, 2023, pp. 1–4.
  12. D. Goldsman and B. L. Nelson, “Statistical screening, selection, and multiple comparison procedures in computer simulation,” in 1998 Winter Simulation Conference. Proceedings (Cat. No. 98CH36274), vol. 1.   IEEE, 1998, pp. 159–166.
  13. M. C. Fu, “Optimization for simulation: Theory vs. practice,” INFORMS Journal on Computing, vol. 14, no. 3, pp. 192–215, 2002.
  14. C.-H. Chen, D. He, M. Fu, and L. H. Lee, “Efficient simulation budget allocation for selecting an optimal subset,” INFORMS Journal on Computing, vol. 20, no. 4, pp. 579–595, 2008.
  15. J. Boesel, B. L. Nelson, and S.-H. Kim, “Using ranking and selection to “clean up” after simulation optimization,” Operations Research, vol. 51, no. 5, pp. 814–825, 2003.
  16. C.-H. Chen, J. Lin, E. Yücesan, and S. E. Chick, “Simulation budget allocation for further enhancing the efficiency of ordinal optimization,” Discrete Event Dynamic Systems, vol. 10, pp. 251–270, 2000.
  17. Y. Rinott, “On two-stage selection procedures and related probability-inequalities,” Communications in Statistics-Theory and methods, vol. 7, no. 8, pp. 799–811, 1978.
  18. Y. Peng, C.-H. Chen, M. C. Fu, and J.-Q. Hu, “Dynamic sampling allocation and design selection,” INFORMS Journal on Computing, vol. 28, no. 2, pp. 195–208, 2016.
  19. Z. Shi, Y. Peng, L. Shi, C.-H. Chen, and M. C. Fu, “Dynamic sampling allocation under finite simulation budget for feasibility determination,” INFORMS Journal on Computing, vol. 34, no. 1, pp. 557–568, 2022.
  20. H. Guclu, S. Kumar, D. Galloway, M. Krauland, R. Sood, A. Bocour, T. B. Hershey, E. van Nostrand, and M. Potter, “An agent-based model for addressing the impact of a disaster on access to primary care services,” Disaster medicine and public health preparedness, vol. 10, no. 3, pp. 386–393, 2016.
  21. W. Wheaton, “US synthetic population database 2005–2009: Quick start guide. rti international; 2012.”
  22. H. Jalal, J. M. Buchanich, M. S. Roberts, L. C. Balmert, K. Zhang, and D. S. Burke, “Changing dynamics of the drug overdose epidemic in the united states from 1979 through 2016,” Science, vol. 361, no. 6408, p. eaau1184, 2018.
  23. A. A. Ahmed, M. A. Rahimian, and M. S. Roberts, “Inferring epidemic dynamics using gaussian process emulation of agent-based simulations,” in Proceedings of Winter Simulation Conference (WSC).   IEEE, 2023.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets