Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
194 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Intensity-free Convolutional Temporal Point Process: Incorporating Local and Global Event Contexts (2306.14072v1)

Published 24 Jun 2023 in cs.LG, cs.AI, and cs.SI

Abstract: Event prediction in the continuous-time domain is a crucial but rather difficult task. Temporal point process (TPP) learning models have shown great advantages in this area. Existing models mainly focus on encoding global contexts of events using techniques like recurrent neural networks (RNNs) or self-attention mechanisms. However, local event contexts also play an important role in the occurrences of events, which has been largely ignored. Popular convolutional neural networks, which are designated for local context capturing, have never been applied to TPP modelling due to their incapability of modelling in continuous time. In this work, we propose a novel TPP modelling approach that combines local and global contexts by integrating a continuous-time convolutional event encoder with an RNN. The presented framework is flexible and scalable to handle large datasets with long sequences and complex latent patterns. The experimental result shows that the proposed model improves the performance of probabilistic sequential modelling and the accuracy of event prediction. To our best knowledge, this is the first work that applies convolutional neural networks to TPP modelling.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (49)
  1. Hawkes processes in finance. Market Microstructure and Liquidity, 1(01):1550005, 2015.
  2. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271, 2018.
  3. Ctrec: A long-short demands evolution model for continuous-time recommendation. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 675–684, 2019.
  4. The million song dataset. In Proceedings of the 12th International Conference on Music Information Retrieval (ISMIR 2011), 2011.
  5. Empirical evaluation of gated recurrent neural networks on sequence modeling. In NIPS 2014 Workshop on Deep Learning, December 2014, 2014.
  6. Event-based incremental recommendation via factors mixed hawkes process. Information Sciences, 639:119007, 2023. ISSN 0020-0255.
  7. An Introduction to the Theory of Point Processes. Volume II: General Theory and Structure. Springer, 2008.
  8. Language modeling with gated convolutional networks. In International conference on machine learning, pages 933–941. PMLR, 2017.
  9. Learning dynamic context graphs for predicting social events. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1007–1016, 2019.
  10. Dynamic knowledge graph based multi-event forecasting. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1585–1595, 2020.
  11. Recurrent marked temporal point processes: Embedding event history to vector. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1555–1564, 2016.
  12. Machine learning methods for earthquake prediction: A survey. In Proceedings of the Fourth Conference on Software Engineering and Information Management (SEIM-2019), Saint Petersburg, Russia, volume 13, page 25, 2019.
  13. A convolutional encoder model for neural machine translation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 123–135, 2017a.
  14. Convolutional sequence to sequence learning. In International conference on machine learning, pages 1243–1252. PMLR, 2017b.
  15. Conformer: Convolution-augmented transformer for speech recognition. In Proceedings of the Annual Conference of the International Speech Communication Association, pages 5036–5040, 2020.
  16. Alan G Hawkes. Point spectra of some mutually exciting point processes. Journal of the Royal Statistical Society: Series B (Methodological), 33(3):438–443, 1971.
  17. A hazard based approach to user return time prediction. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 1719–1728, 2014.
  18. Just in time recommendations: Modeling the dynamics of boredom in activity streams. In Proceedings of the eighth ACM international conference on web search and data mining, pages 233–242, 2015.
  19. Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT, pages 4171–4186, 2019.
  20. Adam: A method for stochastic optimization. In International Conference on Learning Representations, 2015.
  21. Backpropagation applied to handwritten zip code recognition. Neural computation, 1(4):541–551, 1989.
  22. SNAP Datasets: Stanford large network dataset collection. http://snap.stanford.edu/data, June 2014.
  23. Exploring generative neural temporal point process. Transactions on Machine Learning Research, 2022. ISSN 2835-8856.
  24. Spacecraft anomaly detection with attention temporal convolution networks. Neural Computing and Applications, pages 1–9, 2023a.
  25. That-net: Two-layer hidden state aggregation based two-stream network for traffic accident prediction. Information Sciences, 634:744–760, 2023b. ISSN 0020-0255.
  26. Global-local mutual attention model for text classification. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 27(12):2127–2139, 2019.
  27. The neural hawkes process: A neurally self-modulating multivariate point process. Advances in neural information processing systems, 30, 2017.
  28. Fully neural network based model for general temporal point processes. Advances in neural information processing systems, 32, 2019.
  29. A variational point process model for social event sequences. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 173–180, 2020.
  30. Alex Reinhart. A review of self-exciting spatio-temporal point processes and their applications. Statistical Science, 33(3):299–318, 2018.
  31. Ckconv: Continuous kernel convolution for sequential data. In International Conference on Learning Representations, 2021.
  32. Ronald Rosenfeld. Two decades of statistical language modeling: Where do we go from here? Proceedings of the IEEE, 88(8):1270–1278, 2000.
  33. Think globally, act locally: A deep neural network approach to high-dimensional time series forecasting. Advances in neural information processing systems, 32, 2019.
  34. Intensity-free learning of temporal point processes. In International Conference on Learning Representations, 2019.
  35. Implicit neural representations with periodic activation functions. Advances in Neural Information Processing Systems, 33:7462–7473, 2020.
  36. Unipoint: Universally approximating point processes intensities. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 9685–9694, 2021.
  37. Adaptive global-local context fusion for multi-turn spoken language understanding. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 12622–12628, 2022.
  38. Wavenet: A generative model for raw audio. In 9th ISCA Speech Synthesis Workshop, pages 125–125, 2016.
  39. Attention is all you need. Advances in neural information processing systems, 30, 2017.
  40. Long- and short-term preference learning for next poi recommendation. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, page 2301–2304, New York, NY, USA, 2019. Association for Computing Machinery.
  41. Modeling the intensity function of point process via recurrent neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 31, 2017.
  42. Transformer embeddings of irregularly spaced events and their participants. In Proceedings of the Tenth International Conference on Learning Representations, 2022.
  43. Attention-based context-aware sequential recommendation model. Information Sciences, 510:122–134, 2020. ISSN 0020-0255.
  44. Self-attentive hawkes process. In International conference on machine learning, pages 11183–11193. PMLR, 2020.
  45. Multi-task learning for spatio-temporal event forecasting. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pages 1503–1512, 2015a.
  46. Seismic: A self-exciting point process model for predicting tweet popularity. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pages 1513–1522, 2015b.
  47. Deep fourier kernel for self-attentive point processes. In International Conference on Artificial Intelligence and Statistics, pages 856–864. PMLR, 2021.
  48. Dynamic global structure enhanced multi-channel graph neural network for session-based recommendation. Information Sciences, 2022. ISSN 0020-0255.
  49. Transformer hawkes process. In International conference on machine learning, pages 11692–11702. PMLR, 2020.
Citations (10)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now