2000 character limit reached
VT-MRF-SPF: Variable Target Markov Random Field Scalable Particle Filter (2404.18857v1)
Published 29 Apr 2024 in stat.ME, stat.AP, and stat.CO
Abstract: Markov random fields (MRFs) are invaluable tools across diverse fields, and spatiotemporal MRFs (STMRFs) amplify their effectiveness by integrating spatial and temporal dimensions. However, modeling spatiotemporal data introduces additional hurdles, including dynamic spatial dimensions and partial observations, prevalent in scenarios like disease spread analysis and environmental monitoring. Tracking high-dimensional targets with complex spatiotemporal interactions over extended periods poses significant challenges in accuracy, efficiency, and computational feasibility. To tackle these obstacles, we introduce the variable target MRF scalable particle filter (VT-MRF-SPF), a fully online learning algorithm designed for high-dimensional target tracking over STMRFs with varying dimensions under partial observation. We rigorously guarantee algorithm performance, explicitly indicating overcoming the curse of dimensionality. Additionally, we provide practical guidelines for tuning graphical parameters, leading to superior performance in extensive examinations.
- Curse-of-dimensionality revisited: Collapse of the particle filter in very large scale systems. In Probability and statistics: Essays in honor of David A. Freedman, pages 316–334. Institute of Mathematical Statistics, 2008.
- An introduction to sequential Monte Carlo, volume 4. Springer, 2020.
- Computational Doob h-transforms for online filtering of discretely observed diffusions. In International Conference on Machine Learning, pages 5904–5923. PMLR, 2023.
- George Christakos. Spatiotemporal random fields: theory and applications. Elsevier, 2017.
- Sequential Monte Carlo samplers. Journal of the Royal Statistical Society Series B: Statistical Methodology, 68(3):411–436, 2006.
- fMRI signal restoration using a spatio-temporal Markov random field preserving transitions. NeuroImage, 8(4):340–349, 1998.
- An introduction to sequential Monte Carlo methods. Sequential Monte Carlo methods in practice, pages 3–14, 2001.
- Approximate smoothing and parameter estimation in high-dimensional state-space models. IEEE Transactions on Signal Processing, 65(22):5982–5994, 2017.
- Conditional sequential Monte Carlo in high dimensions. The Annals of Statistics, 51(2):437–463, 2023.
- Hans-Otto Georgii. Gibbs measures and phase transitions, volume 9. Walter de Gruyter, 2011.
- Spatiotemporal blocking of the bouncy particle sampler for efficient inference in state space models. arXiv preprint arXiv:2101.03079, 2021.
- The iterated auxiliary particle filter. Journal of the American Statistical Association, 112(520):1636–1647, 2017.
- An iterated block particle filter for inference on coupled dynamic systems with shared and unit-specific parameters. Statistica Sinica, 34:1–22, 2024.
- Data-driven event diagnosis in transmission systems with incomplete and conflicting alarms given sensor malfunctions. IEEE Transactions on Power Delivery, 35(1):214–225, 2019.
- MCMC-based particle filtering for tracking a variable number of interacting targets. IEEE transactions on pattern analysis and machine intelligence, 27(11):1805–1819, 2005.
- Spatio-temporal areal unit modeling in R with conditional autoregressive priors using the CARBayesST package. Journal of Statistical Software, 84:1–39, 2018.
- Estimation of disease rates in small areas: a new mixed model for spatial dependence. In Statistical models in epidemiology, the environment, and clinical trials, pages 179–191. Springer, 2000.
- Inference on spatiotemporal dynamics for networks of biological populations. arXiv preprint arXiv:2311.06702, 2023.
- Substantial undocumented infection facilitates the rapid dissemination of novel coronavirus (SARS-CoV-2). Science, 368(6490):489–493, 2020.
- Stan Z Li. Markov random field modeling in image analysis. Springer Science & Business Media, 2009.
- Stan Z Li. Markov random field modeling in computer vision. Springer Science & Business Media, 2012.
- Statistical machine learning meets high-dimensional spatiotemporal challenges–a case study of COVID-19 modeling. arXiv preprint arXiv:2312.14161, 2023.
- Understanding human-COVID-19 dynamics using geospatial big data: A systematic literature review. arXiv preprint arXiv:2404.10013, 2024.
- Information reconstruction on an infinite tree for a 4×4444\times 44 × 4-state asymmetric model with community effects. Journal of Statistical Physics, 177(3):438–467, 2019a.
- Large degree asymptotics and the reconstruction threshold of the asymmetric binary channels. Journal of Statistical Physics, 174(6):1161–1188, 2019b.
- Phase transition of the reconstructability of a general model with different in-community and out-community mutations on an infinite tree. SIAM Journal on Discrete Mathematics, 35(2):1381–1417, 2021.
- The tightness of the Kesten–Stigum reconstruction bound of symmetric model with multiple mutations. Journal of Statistical Physics, 170(3):617–641, 2018.
- KV Mardia. Multi-dimensional multivariate Gaussian Markov random fields with application to image processing. Journal of Multivariate Analysis, 24(2):265–284, 1988.
- A model to estimate the impact of changes in MMR vaccine uptake on inequalities in measles susceptibility in Scotland. Statistical Methods in Medical Research, 25(4):1185–1200, 2016.
- Iterated block particle filter for high-dimensional parameter learning: Beating the curse of dimensionality. Journal of Machine Learning Research, 24(82):1–76, 2023.
- Non-separable spatio-temporal models via transformed multivariate Gaussian Markov random fields. Journal of the Royal Statistical Society Series C: Applied Statistics, 71(5):1116–1136, 2022.
- Can local particle filters beat the curse of dimensionality? The Annals of Applied Probability, 25(5):2809–2866, 2015.
- Exploiting locality in high-dimensional factorial hidden Markov models. Journal of Machine Learning Research, 23(4):1–34, 2022.
- Gaussian Markov random fields: theory and applications. CRC press, 2005.
- Blocking strategies and stability of particle Gibbs samplers. Biometrika, 104(4):953–969, 2017.
- Obstacles to high-dimensional particle filtering. Monthly Weather Review, 136(12):4629–4640, 2008.
- Quantifying asymptomatic infection and transmission of COVID-19 in New York City using observed cases, serology, and testing capacity. Proceedings of the National Academy of Sciences, 118(9):e2019716118, 2021.