Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
184 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Decision-Dependent Distributionally Robust Markov Decision Process Method in Dynamic Epidemic Control (2306.14051v1)

Published 24 Jun 2023 in math.OC, cs.AI, and cs.LG

Abstract: In this paper, we present a Distributionally Robust Markov Decision Process (DRMDP) approach for addressing the dynamic epidemic control problem. The Susceptible-Exposed-Infectious-Recovered (SEIR) model is widely used to represent the stochastic spread of infectious diseases, such as COVID-19. While Markov Decision Processes (MDP) offers a mathematical framework for identifying optimal actions, such as vaccination and transmission-reducing intervention, to combat disease spreading according to the SEIR model. However, uncertainties in these scenarios demand a more robust approach that is less reliant on error-prone assumptions. The primary objective of our study is to introduce a new DRMDP framework that allows for an ambiguous distribution of transition dynamics. Specifically, we consider the worst-case distribution of these transition probabilities within a decision-dependent ambiguity set. To overcome the computational complexities associated with policy determination, we propose an efficient Real-Time Dynamic Programming (RTDP) algorithm that is capable of computing optimal policies based on the reformulated DRMDP model in an accurate, timely, and scalable manner. Comparative analysis against the classic MDP model demonstrates that the DRMDP achieves a lower proportion of infections and susceptibilities at a reduced cost.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (45)
  1. Learning to act using real-time dynamic programming. Artificial Intelligence, 72(1):81 – 138.
  2. Distributionally robust facility location problem under decision-dependent stochastic demand. European Journal of Operational Research, 292(2):548–561.
  3. Robust lock-down optimization for COVID-19 policy guidance. In AAAI Fall Symposium.
  4. Distributionally robust optimization for sequential decision-making. Optimization, 68(12):2397–2426.
  5. Davies, S. (1996). Multidimensional triangulation and interpolation for reinforcement learning. In Advances in Neural Information Processing Systems, page 1005–1011.
  6. Optimal control applied to vaccination and treatment strategies for various epidemiological models. Mathematical Biosciences & Engineering, 6(3):469–492.
  7. Modelling the COVID-19 epidemic and implementation of population-wide interventions in Italy. Nature Medicine, 26:855–860.
  8. Extensions of the SEIR model for the analysis of tailored social distancing and tracing approaches to cope with COVID-19. Scientific Reports, 11(4214).
  9. Solving mixed integer bilinear problems using MILP formulations. SIAM Journal on Optimization, 23(2):721–744.
  10. Planning Production, Inventories, and Work Force. Prentice Hall.
  11. The effectiveness of quarantine of Wuhan city against the Corona Virus Disease 2019 (COVID-19): A well-mixed SEIR model analysis. Journal of Medical Virology, 92:841–848.
  12. IMF (2021). World economic outlook Oct 2021. International Monetary Fund.
  13. Iyengar, G. N. (2005). Robust dynamic programming. Mathematics of Operations Research, 30(2):257–280.
  14. Learning from SARS: Preparing for the Next Disease Outbreak. The National Academies Press.
  15. Modelling the COVID-19 epidemic and implementation of population-wide interventions in Italy. Biometrics, 62(4):1170–1177.
  16. Distributionally robust optimization with decision dependent ambiguity sets. In Optimization Letters.
  17. A modified SEIR model to predict the COVID-19 outbreak in Spain and Italy: Simulating control scenarios and multi-scale epidemics. Results in Physics, 21:103746.
  18. Planning with Markov Decision Processes: An AI Perspective. Morgan Claypool.
  19. McCormick, G. P. (1976). Computability of global solutions to factorable nonconvex programs: Part I — Convex underestimating problems. Mathematical Programming, 10:147–175.
  20. Moore, D. W. (1992). Simplicial mesh generation with applications. PhD Thesis, Cornell University, Department of Computer Science.
  21. Variable resolution discretization in optimal control. Machine Learning, 49:291–323.
  22. Distributionally robust partially observable Markov decision process with moment-based ambiguity. SIAM Journal on Optimization, 31(1):461–488.
  23. Robustness in Markov decision problems with uncertain transition matrices. In Advances in Neural Information Processing Systems, pages 839–846.
  24. Robust control of Markov decision processes with uncertain transition matrice. Operations Research, 53(5):780–798.
  25. Nouri, A. (2011). Efficient Model-based Exploration in Continuous State-space Environments. Rutgers The State University of New Jersey-New Brunswick.
  26. Distributionally robust optimization under a decision-dependent ambiguity set with applications to machine scheduling and humanitarian logistics. INFORMS Journal on Computing, 34(2):729–751.
  27. Osogami, T. (2012). Robustness and risk-sensitivity in Markov decision processes. In Advances in Neural Information Processing Systems, pages 233–241.
  28. An analytic framework to develop policies for testing, prevention, and treatment of two-stage contagious diseases. Annals of Operations Research, 196(1):707–735.
  29. Pearl, J. (1984). Heuristics: Intelligent Search Strategies for Computer Problem Solving. Addison-Wesley Longman Publishing Co., Inc.
  30. Epidemic analysis of COVID-19 in China by dynamical modeling. Arxiv Preprint ArXiv:2002.06563.
  31. Variational theory for optimization under stochastic ambiguity. SIAM Journal on Optimization, 27(2):1118–1149.
  32. A tractable leader-follower MDP model for animal disease management. In Proceedings of the Twenty-Seventh AAAI Conference on Artificial Intelligence, page 1320–1326.
  33. Only strict quarantine measures can curb the coronavirus disease (COVID-19) outbreak in Italy, 2020. Eurosurveillance, 25(13).
  34. The effectiveness of quarantine and isolation determine the trend of the COVID-19 epidemics in the final phase of the current outbreak in China. International Journal of Infectious Diseases, 95:288–293.
  35. A modelling framework based on MDP to coordinate farmers’ disease control decisions at a regional scale. PLOS ONE, 13(6):1–20.
  36. Markov decision processes with imprecise transition probabilities. Operations Research, 42(4):739–749.
  37. Distributionally robust Markov decision processes. In Advances in Neural Information Processing Systems, pages 2505–2513.
  38. Dynamic health policies for controlling the spread of emerging infections: Influenza as an example. PLOS ONE, 6(9):1–11.
  39. Optimal and sub-optimal quarantine and isolation control in sars epidemics. Mathematical and Computer Modelling, 47(1):235–245.
  40. Yang, I. (2017a). A convex optimization approach to distributionally robust Markov decision processes with Wasserstein distance. IEEE Control Systems Letters, 1(1):164–169.
  41. Yang, I. (2017b). A dynamic game approach to distributionally robust safety specifications for stochastic systems. Automatica, 94:94–101.
  42. Distributionally robust counterpart in Markov decision processes. IEEE Transactions on Automatic Control, 61(9):2538 – 2543.
  43. Multistage distributionally robust mixed-integer programming with decision-dependent moment-based ambiguity sets. Mathematical Programming, 196(1–2):1025–1064.
  44. Quantitative stability analysis for distributionally robust optimization with moment constraints. SIAM Journal on Optimization, 26:1855–1882.
  45. Modeling the epidemic dynamics and control of COVID-19 outbreak in China. Quantative Biology, pages 1–9.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now