Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
175 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Add and Thin: Diffusion for Temporal Point Processes (2311.01139v2)

Published 2 Nov 2023 in cs.LG and stat.ML

Abstract: Autoregressive neural networks within the temporal point process (TPP) framework have become the standard for modeling continuous-time event data. Even though these models can expressively capture event sequences in a one-step-ahead fashion, they are inherently limited for long-term forecasting applications due to the accumulation of errors caused by their sequential nature. To overcome these limitations, we derive ADD-THIN, a principled probabilistic denoising diffusion model for TPPs that operates on entire event sequences. Unlike existing diffusion approaches, ADD-THIN naturally handles data with discrete and continuous components. In experiments on synthetic and real-world datasets, our model matches the state-of-the-art TPP models in density estimation and strongly outperforms them in forecasting.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (52)
  1. J. M. L. Alcaraz and N. Strodthoff. Diffusion-based time series imputation and forecasting with structured state space models. arXiv preprint arXiv:2208.09399, 2022.
  2. Structured denoising diffusion models in discrete state-spaces. Advances in Neural Information Processing Systems, 34:17981–17993, 2021.
  3. Modeling temporal data as continuous functions with stochastic process diffusion. In International Conference on Machine Learning (ICML), 2023.
  4. T. Bosser and S. B. Taieb. On the predictive accuracy of neural temporal point process models for continuous-time event data. Transactions on Machine Learning Research, 2023.
  5. Neural spatio-temporal point processes. arXiv preprint arXiv:2011.04583, 2020.
  6. T. Chen and M. Zhou. Learning to jump: Thinning and thickening latent counts for generative modeling. arXiv preprint arXiv:2305.18375, 2023.
  7. D. R. Cox. Some statistical methods connected with series of events. Journal of the Royal Statistical Society: Series B (Methodological), 17(2):129–157, 1955.
  8. D. Daley and D. Vere-Jones. An Introduction to the Theory of Point Processes: Volume I: Elementary Theory and Methods. Probability and Its Applications. Springer New York, 2006.
  9. D. J. Daley and D. Vere-Jones. An introduction to the theory of point processes: volume II: general theory and structure. Springer Science & Business Media, 2007.
  10. Using deep learning for flexible and scalable earthquake forecasting. Geophysical Research Letters, 50(17), 2023.
  11. C. DR and I. COLL. The statistical analysis of dependencies in point processes. Stochastic Point Processes. Wiley: New York, pages 55–66, 1972.
  12. Recurrent marked temporal point processes: Embedding event history to vector. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1555–1564, 2016.
  13. Neural temporal point processes for modelling electronic health records. In Machine Learning for Health, pages 85–113. PMLR, 2020.
  14. A kernel two-sample test. The Journal of Machine Learning Research, 13(1):723–773, 2012.
  15. Card: Classification and regression diffusion models. arXiv preprint arXiv:2206.07275, 2022.
  16. Inference and sampling of point processes from diffusion excursions. In The 39th Conference on Uncertainty in Artificial Intelligence, 2023.
  17. A. G. Hawkes. Spectra of some self-exciting and mutually exciting point processes. Biometrika, 58(1):83–90, 1971.
  18. Denoising diffusion probabilistic models. Neural Information Processing Systems (NeurIPS), 2020.
  19. C. Hong and C. Shelton. Deep neyman-scott processes. In Proceedings of the 25th International Conference on Artificial Intelligence and Statistics, volume 151, pages 3627–3646. PMLR, 2022.
  20. V. Isham and M. Westcott. A self-correcting point process. Stochastic processes and their applications, 8(3):335–347, 1979.
  21. J. Jia and A. R. Benson. Neural jump stochastic differential equations. Advances in Neural Information Processing Systems, 32, 2019.
  22. D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  23. Turbulent flow simulation using autoregressive conditional diffusion models, 2023.
  24. Predict, refine, synthesize: Self-guiding diffusion models for probabilistic time series forecasting. arXiv preprint arXiv:2307.11494, 2023.
  25. Srdiff: Single image super-resolution with diffusion probabilistic models. Neurocomputing, 479:47–59, 2022a.
  26. Learning temporal point processes via reinforcement learning. Advances in neural information processing systems, 31, 2018.
  27. Diffusion-lm improves controllable text generation. Advances in Neural Information Processing Systems, 35:4328–4343, 2022b.
  28. From zero to turbulence: Generative modeling for 3d flow simulation, 2023.
  29. Exploring generative neural temporal point process. Transactions on Machine Learning Research, 2022.
  30. Pde-refiner: Achieving accurate long rollouts with neural pde solvers. arXiv preprint arXiv:2308.05732, 2023.
  31. S. Luo and W. Hu. Diffusion probabilistic models for 3d point cloud generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2837–2845, 2021.
  32. A conditional point diffusion-refinement paradigm for 3d point cloud completion. arXiv preprint arXiv:2112.03530, 2021.
  33. H. Mei and J. M. Eisner. The neural hawkes process: A neurally self-modulating multivariate point process. In Neural Information Processing Systems (NeurIPS), 2017.
  34. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:2112.10741, 2021.
  35. A. Q. Nichol and P. Dhariwal. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning, pages 8162–8171. PMLR, 2021.
  36. Deep mixture point processes: Spatio-temporal event prediction with rich contextual information. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 373–383, 2019.
  37. Fully neural network based model for general temporal point processes. Advances in neural information processing systems, 32, 2019.
  38. On wasserstein two-sample testing and related families of nonparametric tests. Entropy, 19(2):47, 2017.
  39. B. Sevast’yanov. Renewal theory. Journal of Soviet Mathematics, 4(3):281–302, 1975.
  40. Identifying coordinated accounts on social media through hidden influence and group behaviours. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 1441–1451, 2021.
  41. Intensity-free learning of temporal point processes. In International Conference on Learning Representations (ICLR), 2020a.
  42. Fast and flexible temporal point processes with triangular maps. In Advances in Neural Information Processing Systems (NeurIPS), 2020b.
  43. Neural temporal point processes: A review. arXiv preprint arXiv:2104.03528, 2021.
  44. Unipoint: Universally approximating point processes intensities. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 9685–9694, 2021.
  45. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning (ICML), pages 2256–2265, 2015.
  46. Csdi: Conditional score-based diffusion models for probabilistic time series imputation. Advances in Neural Information Processing Systems, 34:24804–24816, 2021.
  47. Attention is all you need. Advances in neural information processing systems, 30, 2017.
  48. Digress: Discrete denoising diffusion for graph generation. arXiv preprint arXiv:2209.14734, 2022.
  49. Wasserstein learning of deep generative point process models. Advances in neural information processing systems, 30, 2017.
  50. Self-attentive hawkes process. In International conference on machine learning, pages 11183–11193. PMLR, 2020a.
  51. Cause: Learning granger causality from event sequences using attribution methods. In International Conference on Machine Learning, pages 11235–11245. PMLR, 2020b.
  52. Transformer hawkes process. arXiv preprint arXiv:2002.09291, 2020.
Citations (8)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets