Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

DURENDAL: Graph deep learning framework for temporal heterogeneous networks (2310.00336v1)

Published 30 Sep 2023 in cs.LG

Abstract: Temporal heterogeneous networks (THNs) are evolving networks that characterize many real-world applications such as citation and events networks, recommender systems, and knowledge graphs. Although different Graph Neural Networks (GNNs) have been successfully applied to dynamic graphs, most of them only support homogeneous graphs or suffer from model design heavily influenced by specific THNs prediction tasks. Furthermore, there is a lack of temporal heterogeneous networked data in current standard graph benchmark datasets. Hence, in this work, we propose DURENDAL, a graph deep learning framework for THNs. DURENDAL can help to easily repurpose any heterogeneous graph learning model to evolving networks by combining design principles from snapshot-based and multirelational message-passing graph learning models. We introduce two different schemes to update embedding representations for THNs, discussing the strengths and weaknesses of both strategies. We also extend the set of benchmarks for TNHs by introducing two novel high-resolution temporal heterogeneous graph datasets derived from an emerging Web3 platform and a well-established e-commerce website. Overall, we conducted the experimental evaluation of the framework over four temporal heterogeneous network datasets on future link prediction tasks in an evaluation setting that takes into account the evolving nature of the data. Experiments show the prediction power of DURENDAL compared to current solutions for evolving and dynamic graphs, and the effectiveness of its model design.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (65)
  1. Heterogeneous graph neural networks analysis: a survey of techniques, evaluations and applications. Artificial Intelligence Review, pages 1–40, 2022.
  2. Hierarchical temporal convolutional networks for dynamic recommender systems. In The World Wide Web Conference, WWW ’19, page 2236–2246, New York, NY, USA, 2019. Association for Computing Machinery.
  3. Temporal knowledge graph completion: a survey. arXiv preprint arXiv:2201.08236, 2022.
  4. Link prediction with text in online social networks: The role of textual content on high-resolution temporal data. In Discovery Science: 25th International Conference, DS 2022, Montpellier, France, October 10–12, 2022, Proceedings, page 212–226. Springer-Verlag, 2022.
  5. Roland: Graph learning framework for dynamic graphs. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD ’22, page 2358–2366, New York, NY, USA, 2022. Association for Computing Machinery.
  6. Graph neural networks for temporal graphs: State of the art, open challenges, and opportunities. arXiv preprint arXiv:2302.01018, 2023.
  7. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555, 2014.
  8. Spatial temporal incidence dynamic graph neural networks for traffic flow forecasting. Information Sciences, 521:277–290, 2020.
  9. T-GCN: A temporal graph convolutional network for traffic prediction. IEEE Transactions on Intelligent Transportation Systems, 21(9):3848–3858, sep 2020.
  10. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. arXiv preprint arXiv:1707.01926, 2017.
  11. Structured sequence modeling with graph convolutional recurrent networks. In Neural Information Processing: 25th International Conference, ICONIP 2018, Siem Reap, Cambodia, December 13-16, 2018, Proceedings, Part I 25, pages 362–373. Springer, 2018.
  12. Evolvegcn: Evolving graph convolutional networks for dynamic graphs. In Proceedings of the AAAI conference on artificial intelligence, pages 5363–5370, 2020.
  13. Inductive representation learning on temporal graphs. arXiv preprint arXiv:2002.07962, 2020.
  14. Neighborhood-aware scalable temporal network representation learning. In Bastian Rieck and Razvan Pascanu, editors, Proceedings of the First Learning on Graphs Conference, volume 198 of Proceedings of Machine Learning Research, pages 1:1–1:18. PMLR, 09–12 Dec 2022.
  15. Tgl: A general framework for temporal gnn training on billion-scale graphs. Proc. VLDB Endow., 15(8):1572–1580, apr 2022.
  16. Recurrent event network: Autoregressive structure inferenceover temporal knowledge graphs. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6669–6683, Online, November 2020. Association for Computational Linguistics.
  17. Temporal knowledge graph reasoning based on evolutional representation learning. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2021.
  18. Heterogeneous graph transformer. In Proceedings of The Web Conference 2020, WWW ’20, page 2704–2710, New York, NY, USA, 2020. Association for Computing Machinery.
  19. Learning to sample and aggregate: Few-shot reasoning over temporal knowledge graphs. In NeurIPS, 2022.
  20. Optimization of graph neural networks: Implicit acceleration by skip connections and more depth. In International Conference on Machine Learning (ICML), 2021.
  21. Refactor gnns: Revisiting factorisation-based models from a message-passing perspective. In NeurIPS, 2022.
  22. Heterogeneous graph attention network. In The World Wide Web Conference, WWW ’19, page 2022–2032, New York, NY, USA, 2019. Association for Computing Machinery.
  23. Machine learning on graphs: A model and comprehensive taxonomy. Journal of Machine Learning Research, 23(89):1–64, 2022.
  24. Continual graph learning: A survey. arXiv preprint arXiv:2301.12230, 2023.
  25. Joint optimization of tree-based index and deep model for recommender systems, 2019.
  26. Neural temporal walks: Motif-aware representation learning on continuous-time dynamic graphs. In Thirty-Sixth Conference on Neural Information Processing Systems, 2022.
  27. SteemOps. In Proceedings of the Eleventh ACM Conference on Data and Application Security and Privacy. ACM, apr 2021.
  28. Modeling polypharmacy side effects with graph convolutional networks. Bioinform., 34(13):i457–i466, 2018.
  29. Flashlight: Scalable link prediction with effective decoders. In The First Learning on Graphs Conference, LOG ’22, 2022.
  30. Complex embeddings for simple link prediction. In ICML, volume 48 of JMLR Workshop and Conference Proceedings, pages 2071–2080. JMLR.org, 2016.
  31. You can teach an old dog new tricks! on training knowledge graph embeddings. In International Conference on Learning Representations, 2020.
  32. Canonical tensor decomposition for knowledge base completion. In ICML, volume 80 of Proceedings of Machine Learning Research, pages 2869–2878. PMLR, 2018.
  33. Leveraging meta-path based context for top- n recommendation with a neural co-attention model. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD ’18, page 1531–1540, New York, NY, USA, 2018. Association for Computing Machinery.
  34. Attention models in graphs: A survey. ACM Transactions on Knowledge Discovery from Data (TKDD), 13(6):1–25, 2019.
  35. Representation learning for dynamic graphs: A survey. J. Mach. Learn. Res., 21:70:1–70:73, 2020.
  36. Graph attention networks. In International Conference on Learning Representations, 2018.
  37. Temporal graph networks for deep learning on dynamic graphs. CoRR, abs/2006.10637, 2020.
  38. Inductive representation learning in temporal networks via causal anonymous walks. In ICLR. OpenReview.net, 2021.
  39. Tensor decompositions for temporal knowledge base completion, 2020.
  40. Comparing apples and oranges? on the evaluation of methods for temporal knowledge graph forecasting. In Danai Koutra, Claudia Plant, Manuel Gomez Rodriguez, Elena Baralis, and Francesco Bonchi, editors, Machine Learning and Knowledge Discovery in Databases: Research Track, pages 533–549, Cham, 2023. Springer Nature Switzerland.
  41. Temporal knowledge graph completion using box embeddings. In AAAI, pages 7779–7787. AAAI Press, 2022.
  42. Temp: Temporal message passing for temporal knowledge graph completion. In EMNLP (1), pages 5730–5746. Association for Computational Linguistics, 2020.
  43. Modeling relational data with graph convolutional networks. In The Semantic Web: 15th International Conference, ESWC 2018, Heraklion, Crete, Greece, June 3–7, 2018, Proceedings 15, pages 593–607. Springer, 2018.
  44. Convolutional neural networks on graphs with fast localized spectral filtering. In Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS’16, page 3844–3852, Red Hook, NY, USA, 2016. Curran Associates Inc.
  45. Neural message passing for quantum chemistry. In ICML, volume 70 of Proceedings of Machine Learning Research, pages 1263–1272. PMLR, 2017.
  46. Temporal smoothness regularisers for neural link predictors. CoRR, abs/2309.09045, 2023.
  47. Fast graph representation learning with PyTorch Geometric. In ICLR 2019 Workshop on Representation Learning on Graphs and Manifolds, 2019.
  48. PyTorch Geometric Temporal: Spatiotemporal Signal Processing with Neural Machine Learning Models. In Proceedings of the 30th ACM International Conference on Information and Knowledge Management, page 4564–4573, 2021.
  49. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun, editors, 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
  50. Inductive representation learning on large graphs. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, page 1025–1035, Red Hook, NY, USA, 2017. Curran Associates Inc.
  51. Weisfeiler and leman go neural: Higher-order graph neural networks. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01):4602–4609, Jul. 2019.
  52. Measuring and relieving the over-smoothing problem for graph neural networks from the topological view. Proceedings of the AAAI Conference on Artificial Intelligence, 34(04):3438–3445, Apr. 2020.
  53. Open graph benchmark: Datasets for machine learning on graphs. Advances in neural information processing systems, 33:22118–22133, 2020.
  54. Temporal graph benchmark for machine learning on temporal graphs. CoRR, abs/2307.01026, 2023.
  55. Towards better evaluation for dynamic link prediction. ArXiv, abs/2207.10128, 2022.
  56. A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems, 32(1):4–24, jan 2021.
  57. metapath2vec: Scalable representation learning for heterogeneous networks. In KDD, pages 135–144. ACM, 2017.
  58. Embedding entities and relations for learning and inference in knowledge bases. In ICLR (Poster), 2015.
  59. William L. Hamilton. Graph Representation Learning. Synthesis Lectures on Artificial Intelligence and Machine Learning. Morgan & Claypool Publishers, 2020.
  60. node2vec: Scalable feature learning for networks. In KDD, pages 855–864. ACM, 2016.
  61. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992, 2019.
  62. Steemit developer documentation. Broadcast Ops, 2021.
  63. Understanding echo chambers in e-commerce recommender systems. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’20, page 2261–2270, New York, NY, USA, 2020. Association for Computing Machinery.
  64. The effect of people recommenders on echo chambers and polarization. In Proceedings of the International AAAI Conference on Web and Social Media, volume 16, pages 90–101, 2022.
  65. A survey on the fairness of recommender systems. ACM Trans. Inf. Syst., 41(3), feb 2023.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now