Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
149 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Enhancing Node Representations for Real-World Complex Networks with Topological Augmentation (2402.13033v2)

Published 20 Feb 2024 in cs.LG, cs.IR, and cs.SI

Abstract: Graph augmentation methods play a crucial role in improving the performance and enhancing generalisation capabilities in Graph Neural Networks (GNNs). Existing graph augmentation methods mainly perturb the graph structures, and are usually limited to pairwise node relations. These methods cannot fully address the complexities of real-world large-scale networks, which often involve higher-order node relations beyond only being pairwise. Meanwhile, real-world graph datasets are predominantly modelled as simple graphs, due to the scarcity of data that can be used to form higher-order edges. Therefore, reconfiguring the higher-order edges as an integration into graph augmentation strategies lights up a promising research path to address the aforementioned issues. In this paper, we present Topological Augmentation (TopoAug), a novel graph augmentation method that builds a combinatorial complex from the original graph by constructing virtual hyperedges directly from the raw data. TopoAug then produces auxiliary node features by extracting information from the combinatorial complex, which are used for enhancing GNN performances on downstream tasks. We design three diverse virtual hyperedge construction strategies to accompany the construction of combinatorial complexes: (1) via graph statistics, (2) from multiple data perspectives, and (3) utilising multi-modality. Furthermore, to facilitate TopoAug evaluation, we provide 23 novel real-world graph datasets across various domains including social media, biology, and e-commerce. Our empirical study shows that TopoAug consistently and significantly outperforms GNN baselines and other graph augmentation methods, across a variety of application contexts, which clearly indicates that it can effectively incorporate higher-order node relations into the graph augmentation for real-world complex networks.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (43)
  1. Hypergraph convolution and hypergraph attention. Pattern Recognition 110 (2021), 107637.
  2. GRAND: a database of gene regulatory network models across human conditions. Nucleic Acids Research 50, D1 (2022), D610–D621.
  3. How attentive are graph attention networks?. In The 10th International Conference on Learning Representations. OpenReview.net, Virtual.
  4. Chen Cai and Yusu Wang. 2020. A note on over-smoothing for graph neural networks. ICML 2020 Graph Representation Learning and Beyond Workshop.
  5. T-H Hubert Chan and Zhibin Liang. 2020. Generalizing the hypergraph laplacian via a diffusion process with mediators. Theoretical Computer Science 806 (2020), 416–428.
  6. MiDaS: Representative sampling from real-world hypergraphs. In Proceedings of the ACM Web Conference 2022. ACM, New York, NY, USA, 1080–1092.
  7. Data augmentation for deep graph learning: A survey. ACM SIGKDD Explorations Newsletter 24, 2 (2022), 61–77.
  8. HNHN: Hypergraph networks with hyperedge neurons. ICML 2020 Graph Representation Learning and Beyond Workshop.
  9. Sampling hypergraphs with given degrees. Discrete Mathematics 344, 11 (2021), 112566.
  10. Deep learning: new computational modelling techniques for genomics. Nature Reviews Genetics 20, 7 (2019), 389–403.
  11. Graph adversarial training: Dynamically regularizing based on graph structure. IEEE Transactions on Knowledge and Data Engineering 33, 6 (2019), 2493–2504.
  12. Hypergraph neural networks. Proceedings of the AAAI Conference on Artificial Intelligence 33, 01 (2019), 3558–3565.
  13. A Closer Look at Deep Learning Heuristics: Learning rate restarts, Warmup and Distillation. In The 7th International Conference on Learning Representations. OpenReview.net, New Orleans, LA, USA.
  14. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems, Vol. 30. Curran Associates, Inc., Red Hook, NY, USA, 1024–1034.
  15. William L. Hamilton. 2020. Graph representation learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, Vol. 14, No. 3. Morgan & Claypool Publishers, Kentfield, CA, USA.
  16. G-Mixup: Graph Data Augmentation for Graph Classification. In Proceedings of the 39th International Conference on Machine Learning, Vol. 162. PMLR, Baltimore, MD, USA, 8230–8248.
  17. Ruining He and Julian McAuley. 2016. Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering. In Proceedings of the 25th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee, Geneva, Switzerland, 507–517.
  18. Open Graph Benchmark: Datasets for Machine Learning on Graphs. In Advances in Neural Information Processing Systems, Vol. 33. Curran Associates, Inc., Red Hook, NY, USA, 22118–22133.
  19. ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision. In Proceedings of the 38th International Conference on Machine Learning, Vol. 139. PMLR, Virtual, 5583–5594.
  20. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In The 3rd International Conference on Learning Representations, Conference Track Proceedings. ICLR, San Diego, CA, USA.
  21. Thomas N Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In The 5th International Conference on Learning Representations, Conference Track Proceedings. OpenReview.net, Toulon, France.
  22. Data-Centric Learning from Unlabeled Graphs with Diffusion Model. In Advances in Neural Information Processing Systems. Curran Associates, Inc., Red Hook, NY, USA, 19 pages.
  23. How do lncRNAs regulate transcription? Science Advances 3, 9 (2017), eaao2110.
  24. Transcriptional regulatory elements in the human genome. Annual Review of Genomics and Human Genetics 7 (2006), 29–59.
  25. Image-Based Recommendations on Styles and Substitutes. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, New York, NY, USA, 43–52.
  26. Fake news detection on social media using geometric deep learning. ICLR 2019 Workshop on Representation Learning on Graphs and Manifolds.
  27. Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. ACL, Hong Kong, China, 188–197.
  28. Learning transferable visual models from natural language supervision. In Proceedings of the 38th International Conference on Machine Learning, Vol. 139. PMLR, Virtual, 8748–8763.
  29. DropEdge: Towards Deep Graph Convolutional Networks on Node Classification. In The 8th International Conference on Learning Representations. OpenReview.net, Virtual.
  30. Multi-Scale attributed node embedding. Journal of Complex Networks 9, 2 (2021), cnab014.
  31. A survey on oversmoothing in graph neural networks. arXiv preprint arXiv:2303.10993.
  32. Pitfalls of graph neural network evaluation. NeurIPS 2018 Relational Representation Learning Workshop.
  33. Attention is all you need. Advances in neural information processing systems 30 (2017), 5998–6008.
  34. Graph attention networks. In The 6th International Conference on Learning Representations, Conference Track Proceedings. OpenReview.net, Vancouver, Canada.
  35. Equivariant Hypergraph Diffusion Neural Operators. In The 11th International Conference on Learning Representations. OpenReview.net, Kigali, Rwanda.
  36. Mixup for Node and Graph Classification. In Proceedings of the Web Conference 2021. ACM, New York, NY, USA, 3663–3674.
  37. Multi-modality cross attention network for image and sentence matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, Seattle, WA, USA, 10941–10950.
  38. Boris Weisfeiler and Andrei Leman. 1968. A reduction of a graph to canonical form and an algebra arising during this reduction. Nauchno-Technicheskaya Informatsia 2, 9 (1968), 12–16. English translation available at https://www.iti.zcu.cz/wl2018/pdf/wl_paper_translation.pdf.
  39. How Powerful are Graph Neural Networks?. In The 7th International Conference on Learning Representations. OpenReview.net, New Orleans, LA, USA.
  40. HyperGCN: A new method for training graph convolutional networks on hypergraphs. In Advances in Neural Information Processing Systems, Vol. 32. Curran Associates, Inc., Red Hook, NY, USA, 1511–1522.
  41. Graph Contrastive Learning with Augmentations. In Advances in Neural Information Processing Systems, Vol. 33. Curran Associates, Inc., Red Hook, NY, USA, 5812–5823.
  42. GraphSAINT: Graph sampling based inductive learning method. In The 8th International Conference on Learning Representations. OpenReview.net, Virtual.
  43. mixup: Beyond Empirical Risk Minimization. In 6th International Conference on Learning Representations, Conference Track Proceedings. OpenReview.net, Vancouver, Canada.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets