Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Probabilistic Modeling for Sequences of Sets in Continuous-Time (2312.15045v3)

Published 22 Dec 2023 in cs.LG and stat.ML

Abstract: Neural marked temporal point processes have been a valuable addition to the existing toolbox of statistical parametric models for continuous-time event data. These models are useful for sequences where each event is associated with a single item (a single type of event or a "mark") -- but such models are not suited for the practical situation where each event is associated with a set of items. In this work, we develop a general framework for modeling set-valued data in continuous-time, compatible with any intensity-based recurrent neural point process model. In addition, we develop inference methods that can use such models to answer probabilistic queries such as "the probability of item $A$ being observed before item $B$," conditioned on sequence history. Computing exact answers for such queries is generally intractable for neural models due to both the continuous-time nature of the problem setting and the combinatorially-large space of potential outcomes for each event. To address this, we develop a class of importance sampling methods for querying with set-based sequences and demonstrate orders-of-magnitude improvements in efficiency over direct sampling via systematic experiments with four real-world datasets. We also illustrate how to use this framework to perform model selection using likelihoods that do not involve one-step-ahead prediction.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (40)
  1. Hawkes model for price and trades high-frequency dynamics. Quantitative Finance, 14(7):1147–1166.
  2. Inference of multivariate exponential Hawkes processes with inhibition and application to neuronal activity. Statistics and Computing, 33(4):91.
  3. Probabilistic querying of continuous-time event sequences. In International Conference on Artificial Intelligence and Statistics, pages 10235–10251. PMLR.
  4. Brillinger, D. R. (1975). The identification of point process systems. The Annals of Probability, pages 909–924.
  5. An Introduction to the Theory of Point Processes: Volume I: Elementary Theory and Methods, page 274. Springer.
  6. Long horizon forecasting with temporal point processes. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining, pages 571–579.
  7. Recurrent marked temporal point processes: Embedding event history to vector. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1555–1564.
  8. Network self-exciting point processes to measure health impacts of COVID-19. Journal of the Royal Statistical Society Series A: Statistics in Society, 186(3):401–421.
  9. Learning temporal point processes with intermittent observations. In International Conference on Artificial Intelligence and Statistics, pages 3790–3798. PMLR.
  10. The MovieLens datasets: History and context. ACM Transactions on Interactive Intelligent Systems (TIIS), 5(4):1–19.
  11. Hawkes, A. G. (1971). Spectra of some self-exciting and mutually exciting point processes. Biometrika, 58(1):83–90.
  12. Hawkes, A. G. (2018). Hawkes processes and their applications to finance: a review. Quantitative Finance, 18(2):193–198.
  13. From viral evolution to spatial contagion: a biologically modulated Hawkes model. Bioinformatics, 38(7):1846–1856.
  14. Sets2sets: Learning from sequential sets with neural networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1491–1499.
  15. Adam: A method for stochastic optimization. International Conference on Learning Representations (ICLR).
  16. Structured determinantal point processes. Advances in Neural Information Processing Systems, 23:1171–1179.
  17. Determinantal point processes for machine learning. Foundations and Trends® in Machine Learning, 5(2–3):123–286.
  18. Predicting dynamic embedding trajectory in temporal interaction networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1269–1278.
  19. Comparison of prospective Hawkes and recursive point process models for Ebola in DRC. Journal of Forecasting, 41(1):201–210.
  20. Bridging recommendation and marketing via recurrent intensity modeling. In International Conference on Learning Representations.
  21. Macchi, O. (1975). The coincidence approach to stochastic point processes. Advances in Applied Probability, 7(1):83–122.
  22. The million song dataset challenge. In Proceedings of the 21st International Conference on World Wide Web, pages 909–916.
  23. The neural Hawkes process: A neurally self-modulating multivariate point process. Advances in Neural Information Processing Systems, 30:6757–6767.
  24. Imputing missing events in continuous-time event streams. In International Conference on Machine Learning, pages 4475–4485. PMLR.
  25. Ogata, Y. (1981). On Lewis’ simulation method for point processes. IEEE Transactions on Information Theory, 27(1):23–31.
  26. Mean-field limits for non-linear Hawkes processes with excitation and inhibition. Stochastic Processes and their Applications, 153:57–78.
  27. Toward accurate spatiotemporal COVID-19 risk scores using high-resolution real-world mobility data. ACM Transactions on Spatial Algorithms and Systems (TSAS), 8(2):1–30.
  28. Factorizing personalized Markov chains for next-basket recommendation. In Proceedings of the 19th International Conference on World Wide Web, pages 811–820.
  29. Intensity-free learning of temporal point processes. In International Conference on Learning Representations.
  30. Neural temporal point processes: A review. arXiv preprint arXiv:2104.03528.
  31. Hawkes process inference with missing data. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, pages 6425–6432.
  32. Concurrent multi-label prediction in event streams. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 9820–9828.
  33. Fastpoint: Scalable deep point processes. In Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases: ECML/PKDD, pages 465–480. Springer.
  34. Van der Vaart, A. W. (2000). Asymptotic Statistics, volume 3. Cambridge University Press.
  35. Continuous time dynamic topic models. In Proceedings of the Twenty-Fourth Conference on Uncertainty in Artificial Intelligence, pages 579–586.
  36. Topics over time: a non-Markov continuous-time model of topical trends. In Proceedings of the 12th ACM SIGKDD international Conference on Knowledge Discovery and Data Mining, pages 424–433.
  37. Classification of flash crashes using the Hawkes (p, q) framework. Quantitative Finance, 22(2):213–240.
  38. Deep determinantal point process for large-scale multi-label classification. In Proceedings of the IEEE International Conference on Computer Vision, pages 473–482.
  39. HYPRO: A hybridly normalized probabilistic model for long-horizon prediction of event sequences. Advances in Neural Information Processing Systems, 35:34641–34650.
  40. A dynamic recurrent model for next basket recommendation. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval, pages 729–732.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now