Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
144 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Spatio-temporal Diffusion Point Processes (2305.12403v2)

Published 21 May 2023 in cs.LG

Abstract: Spatio-temporal point process (STPP) is a stochastic collection of events accompanied with time and space. Due to computational complexities, existing solutions for STPPs compromise with conditional independence between time and space, which consider the temporal and spatial distributions separately. The failure to model the joint distribution leads to limited capacities in characterizing the spatio-temporal entangled interactions given past events. In this work, we propose a novel parameterization framework for STPPs, which leverages diffusion models to learn complex spatio-temporal joint distributions. We decompose the learning of the target joint distribution into multiple steps, where each step can be faithfully described by a Gaussian distribution. To enhance the learning of each step, an elaborated spatio-temporal co-attention module is proposed to capture the interdependence between the event time and space adaptively. For the first time, we break the restrictions on spatio-temporal dependencies in existing solutions, and enable a flexible and accurate modeling paradigm for STPPs. Extensive experiments from a wide range of fields, such as epidemiology, seismology, crime, and urban mobility, demonstrate that our framework outperforms the state-of-the-art baselines remarkably, with an average improvement of over 50%. Further in-depth analyses validate its ability to capture spatio-temporal interactions, which can learn adaptively for different scenarios. The datasets and source code are available online: https://github.com/tsinghua-fib-lab/Spatio-temporal-Diffusion-Point-Processes.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (68)
  1. Structured denoising diffusion models in discrete state-spaces. Advances in Neural Information Processing Systems 34 (2021), 17981–17993.
  2. Spatial point processes and their applications. Stochastic Geometry: Lectures Given at the CIME Summer School Held in Martina Franca, Italy, September 13–18, 2004 (2007), 1–75.
  3. Neural spatio-temporal point processes. (2021).
  4. Neural ordinary differential equations. Advances in neural information processing systems 31 (2018).
  5. An introduction to the theory of point processes: volume I: elementary theory and methods. Springer.
  6. Prafulla Dhariwal and Alexander Nichol. 2021. Diffusion models beat gans on image synthesis. Advances in Neural Information Processing Systems 34 (2021), 8780–8794.
  7. Peter J Diggle. 2006. Spatio-temporal point processes: methods and applications. Monographs on Statistics and Applied Probability 107 (2006), 1.
  8. Wasserstein generative adversarial networks for modeling marked events. The Journal of Supercomputing (2022), 1–23.
  9. Recurrent marked temporal point processes: Embedding event history to vector. In KDD. 1555–1564.
  10. Difformer: Empowering Diffusion Model on Embedding Space for Text Generation. arXiv preprint arXiv:2212.09412 (2022).
  11. It’s raw! audio generation with state-space models. In International Conference on Machine Learning. PMLR, 7616–7633.
  12. Diffuseq: Sequence to sequence text generation with diffusion models. arXiv preprint arXiv:2210.08933 (2022).
  13. Spatio-temporal point process statistics: a review. Spatial Statistics 18 (2016), 505–544.
  14. Jan Grandell. 2012. Aspects of risk theory. Springer Science & Business Media.
  15. Stochastic trajectory prediction via motion indeterminacy diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 17113–17122.
  16. INITIATOR: Noise-contrastive Estimation for Marked Temporal Point Process.. In IJCAI. 2191–2197.
  17. Alan G Hawkes. 1971. Point spectra of some mutually exciting point processes. Journal of the Royal Statistical Society: Series B (Methodological) 33, 3 (1971), 438–443.
  18. Alan G Hawkes. 2018. Hawkes processes and their applications to finance: a review. Quantitative Finance 18, 2 (2018), 193–198.
  19. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems 33 (2020), 6840–6851.
  20. Video diffusion models. arXiv preprint arXiv:2204.03458 (2022).
  21. Valerie Isham and Mark Westcott. 1979. A self-correcting point process. Stochastic processes and their applications 8, 3 (1979), 335–347.
  22. Junteng Jia and Austin R Benson. 2019. Neural jump stochastic differential equations. Advances in Neural Information Processing Systems 32 (2019).
  23. John Frank Charles Kingman. 1992. Poisson processes. Vol. 3. Clarendon Press.
  24. Diffwave: A versatile diffusion model for audio synthesis. arXiv preprint arXiv:2009.09761 (2020).
  25. Automated Spatio-Temporal Synchronous Modeling with Multiple Graphs for Traffic Prediction. In CIKM. 1084–1093.
  26. Learning temporal point processes via reinforcement learning. Advances in neural information processing systems 31 (2018).
  27. Diffusion-lm improves controllable text generation. arXiv preprint arXiv:2205.14217 (2022).
  28. Exploring Generative Neural Temporal Point Process. JMLR (2022).
  29. Practical Synthetic Human Trajectories Generation Based on Variational Point Processes. In KDD 2023.
  30. Shitong Luo and Wei Hu. 2021. Diffusion probabilistic models for 3d point cloud generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2837–2845.
  31. A variational auto-encoder model for stochastic point processes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3165–3174.
  32. Hongyuan Mei and Jason M Eisner. 2017. The neural hawkes process: A neurally self-modulating multivariate point process. Advances in neural information processing systems 30 (2017).
  33. Noise-contrastive estimation for multivariate point processes. Advances in neural information processing systems 33 (2020), 5204–5214.
  34. A space–time conditional intensity model for invasive meningococcal disease occurrence. Biometrics 68, 2 (2012), 607–616.
  35. Jesper Moller and Rasmus Plenge Waagepetersen. 2003. Statistical inference and simulation for spatial point processes. CRC press.
  36. Yosihiko Ogata. 1988. Statistical models for earthquake occurrences and residual analysis for point processes. Journal of the American Statistical association 83, 401 (1988), 9–27.
  37. Deep mixture point processes: Spatio-temporal event prediction with rich contextual information. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 373–383.
  38. Fully neural network based model for general temporal point processes. Advances in neural information processing systems 32 (2019).
  39. A variational point process model for social event sequences. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 173–180.
  40. A non-parametric hawkes model of the spread of ebola in west africa. Journal of Applied Statistics 49, 3 (2022), 621–637.
  41. Variable-drift diffusion models of pedestrian road-crossing decisions. Computational Brain & Behavior (2021), 1–21.
  42. Rüdiger Rackwitz and Bernd Flessler. 1978. Structural reliability under combined random load sequences. Computers & structures 9, 5 (1978), 489–494.
  43. Jakob Gulddahl Rasmussen. 2018. Lecture notes: Temporal point processes and the conditional intensity function. arXiv preprint arXiv:1806.00221 (2018).
  44. Autoregressive denoising diffusion models for multivariate probabilistic time series forecasting. In International Conference on Machine Learning. PMLR, 8857–8868.
  45. Alex Reinhart. 2018. A review of self-exciting spatio-temporal point processes and their applications. Statist. Sci. 33, 3 (2018), 299–318.
  46. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10684–10695.
  47. Intensity-free learning of temporal point processes. (2019).
  48. D2c: Diffusion-decoding models for few-shot conditional generation. Advances in Neural Information Processing Systems 34 (2021), 12533–12548.
  49. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning. PMLR, 2256–2265.
  50. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502 (2020).
  51. CSDI: Conditional score-based diffusion models for probabilistic time series imputation. Advances in Neural Information Processing Systems 34 (2021), 24804–24816.
  52. Deep reinforcement learning of marked temporal point processes. Advances in Neural Information Processing Systems 31 (2018).
  53. Attention is all you need. Advances in neural information processing systems 30 (2017).
  54. Spatio-temporal urban knowledge graph enabled mobility prediction. Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies 5, 4 (2021), 1–24.
  55. Earthquake prediction based on spatio-temporal data mining: an LSTM network approach. IEEE Transactions on Emerging Topics in Computing 8, 1 (2017), 148–158.
  56. Stanisław W\keglarczyk. 2018. Kernel density estimation and its application. In ITM Web of Conferences, Vol. 23. EDP Sciences, 00037.
  57. Wasserstein learning of deep generative point process models. Advances in neural information processing systems 30 (2017).
  58. Crowdsourcing based description of urban emergency events using social media big data. IEEE Transactions on Cloud Computing 8, 2 (2016), 387–397.
  59. Recurrent spatio-temporal point process for check-in time prediction. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management. 2203–2211.
  60. Spatio-Temporal Vehicle Trajectory Recovery on Road Network Based on Traffic Camera Video Data. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 4413–4421.
  61. Activity Trajectory Generation via Modeling Spatiotemporal Dynamics. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 4752–4762.
  62. Learning to Simulate Daily Activities via Modeling Dynamic Human Needs. In Proceedings of the ACM Web Conference 2023. 906–916.
  63. Self-attentive Hawkes process. In ICML. PMLR, 11183–11193.
  64. 3d shape generation and completion through point-voxel diffusion. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 5826–5835.
  65. Neural Point Process for Learning Spatiotemporal Event Dynamics. In Learning for Dynamics and Control Conference. PMLR, 777–789.
  66. Spatio-temporal point processes with attention for traffic congestion event modeling. IEEE TITS 23, 7 (2021), 7298–7309.
  67. Jiancang Zhuang. 2006. Second-order residual analysis of spatiotemporal point processes and applications in model evaluation. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 68, 4 (2006), 635–653.
  68. Transformer hawkes process. In International conference on machine learning. PMLR, 11692–11702.
Citations (32)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets