Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
194 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Integration-free Training for Spatio-temporal Multimodal Covariate Deep Kernel Point Processes (2310.05485v1)

Published 9 Oct 2023 in cs.LG and stat.ML

Abstract: In this study, we propose a novel deep spatio-temporal point process model, Deep Kernel Mixture Point Processes (DKMPP), that incorporates multimodal covariate information. DKMPP is an enhanced version of Deep Mixture Point Processes (DMPP), which uses a more flexible deep kernel to model complex relationships between events and covariate data, improving the model's expressiveness. To address the intractable training procedure of DKMPP due to the non-integrable deep kernel, we utilize an integration-free method based on score matching, and further improve efficiency by adopting a scalable denoising score matching method. Our experiments demonstrate that DKMPP and its corresponding score-based estimators outperform baseline models, showcasing the advantages of incorporating covariate information, utilizing a deep kernel, and employing score-based estimators.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (43)
  1. Including covariates in a space-time point process with application to seismicity. Statistical Methods & Applications, 30(3):947–971, 2021.
  2. Modelling microstructure noise with mutually exciting point processes. Quantitative Finance, 13(1):65–77, 2013.
  3. Nonparametric estimation of the dependence of a spatial point process on spatial covariates. Statistics and its interface, 5(2):221–236, 2012.
  4. Neural spatio-temporal point processes. In International Conference on Learning Representations, 2021.
  5. Cinlar, E. Markov renewal theory. Advances in Applied Probability, 1(2):123–187, 1969.
  6. An introduction to the theory of point processes. vol. i. probability and its applications, 2003.
  7. Recurrent marked temporal point processes: Embedding event history to vector. In Krishnapuram, B., Shah, M., Smola, A. J., Aggarwal, C. C., Shen, D., and Rastogi, R. (eds.), Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, August 13-17, 2016, pp.  1555–1564. ACM, 2016.
  8. Risk theory for the compound poisson process that is perturbed by diffusion. Insurance: mathematics and economics, 10(1):51–59, 1991.
  9. Point process models for covid-19 cases and deaths. Journal of Applied Statistics, pp.  1–16, 2021.
  10. Hawkes, A. G. Spectra of some self-exciting and mutually exciting point processes. Biometrika, 58(1):83–90, 1971.
  11. Hawkes, A. G. Hawkes processes and their applications to finance: a review. Quantitative Finance, 18(2):193–198, 2018.
  12. Hyvärinen, A. Estimation of non-normalized statistical models by score matching. J. Mach. Learn. Res., 6:695–709, 2005.
  13. Kingman, J. F. C. Poisson processes, volume 3. Clarendon Press, 1992.
  14. The neural Hawkes process: A neurally self-modulating multivariate point process. In Guyon, I., von Luxburg, U., Bengio, S., Wallach, H. M., Fergus, R., Vishwanathan, S. V. N., and Garnett, R. (eds.), Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp. 6754–6764, 2017.
  15. A space–time conditional intensity model for invasive meningococcal disease occurrence. Biometrics, 68(2):607–616, 2012.
  16. Mohler, G. Modeling and estimation of multi-source clustering in crime and security data. The Annals of Applied Statistics, pp.  1525–1539, 2013.
  17. Self-exciting point process modeling of crime. Journal of the American Statistical Association, 106(493):100–108, 2011.
  18. Log Gaussian Cox processes. Scandinavian journal of statistics, 25(3):451–482, 1998.
  19. Ogata, Y. Space-time point-process models for earthquake occurrences. Annals of the Institute of Statistical Mathematics, 50(2):379–402, 1998.
  20. Fast likelihood computation of epidemic type aftershock-sequence model. Geophysical research letters, 20(19):2143–2146, 1993.
  21. Deep mixture point processes: Spatio-temporal event prediction with rich contextual information. In Teredesai, A., Kumar, V., Li, Y., Rosales, R., Terzi, E., and Karypis, G. (eds.), Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2019, Anchorage, AK, USA, August 4-8, 2019, pp.  373–383. ACM, 2019.
  22. Dynamic hawkes processes for discovering time-evolving communities’ states behind diffusion processes. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp.  1276–1286, 2021.
  23. Fully neural network based model for general temporal point processes. In Wallach, H. M., Larochelle, H., Beygelzimer, A., d’Alché-Buc, F., Fox, E. B., and Garnett, R. (eds.), Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp.  2120–2129, 2019.
  24. Paninski, L. Maximum likelihood estimation of cascade point-process neural encoding models. Network: Computation in Neural Systems, 15(4):243–262, 2004.
  25. Sir-hawkes: Linking epidemic models and hawkes processes to model diffusions in finite populations. In Champin, P., Gandon, F., Lalmas, M., and Ipeirotis, P. G. (eds.), Proceedings of the 2018 World Wide Web Conference on World Wide Web, WWW 2018, Lyon, France, April 23-27, 2018, pp.  419–428. ACM, 2018.
  26. Score-matching estimators for continuous-time point-process regression models. In Palmieri, F. A. N., Uncini, A., Diamantaras, K. I., and Larsen, J. (eds.), 26th IEEE International Workshop on Machine Learning for Signal Processing, MLSP 2016, Vietri sul Mare, Salerno, Italy, September 13-16, 2016, pp.  1–5. IEEE, 2016.
  27. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter, 2020.
  28. Scargle, J. D. Studies in astronomical time series analysis. v. bayesian blocks, a new method to analyze structure in photon counting data. The Astrophysical Journal, 504(1):405, 1998.
  29. Intensity-free learning of temporal point processes. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020a.
  30. Fast and flexible temporal point processes with triangular maps. Advances in Neural Information Processing Systems, 33:73–84, 2020b.
  31. A point process framework for relating neural spiking activity to spiking history, neural ensemble, and extrinsic covariate effects. Journal of neurophysiology, 93(2):1074–1089, 2005.
  32. Vincent, P. A connection between score matching and denoising autoencoders. Neural Comput., 23(7):1661–1674, 2011.
  33. Waagepetersen, R. Estimating functions for inhomogeneous spatial point processes with incomplete covariate data. Biometrika, 95(2):351–363, 2008.
  34. Poisson point process models solve the" pseudo-absence problem" for presence-only data in ecology. The Annals of Applied Statistics, pp.  1383–1402, 2010.
  35. Deep kernel learning. In Gretton, A. and Robert, C. C. (eds.), Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, AISTATS 2016, Cadiz, Spain, May 9-11, 2016, volume 51 of JMLR Workshop and Conference Proceedings, pp.  370–378. JMLR.org, 2016.
  36. Modeling the intensity function of point process via recurrent neural networks. In Singh, S. and Markovitch, S. (eds.), Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, February 4-9, 2017, San Francisco, California, USA, pp.  1597–1603. AAAI Press, 2017.
  37. Self-attentive Hawkes process. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pp.  11183–11193. PMLR, 2020.
  38. Efficient inference for nonparametric Hawkes processes using auxiliary latent variables. Journal of Machine Learning Research, 21(241):1–31, 2020.
  39. Efficient inference of flexible interaction in spiking-neuron networks. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
  40. Efficient inference for dynamic flexible interactions of neural populations. Journal of Machine Learning Research, 23(211):1–49, 2022a.
  41. Neural point process for learning spatiotemporal event dynamics. In Learning for Dynamics and Control Conference, pp. 777–789. PMLR, 2022b.
  42. Neural spectral marked point processes. In International Conference on Learning Representations, 2021.
  43. Transformer Hawkes process. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pp.  11692–11702. PMLR, 2020.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now