Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Graph Contrastive Learning with Cohesive Subgraph Awareness (2401.17580v2)

Published 31 Jan 2024 in cs.LG and cs.AI

Abstract: Graph contrastive learning (GCL) has emerged as a state-of-the-art strategy for learning representations of diverse graphs including social and biomedical networks. GCL widely uses stochastic graph topology augmentation, such as uniform node dropping, to generate augmented graphs. However, such stochastic augmentations may severely damage the intrinsic properties of a graph and deteriorate the following representation learning process. We argue that incorporating an awareness of cohesive subgraphs during the graph augmentation and learning processes has the potential to enhance GCL performance. To this end, we propose a novel unified framework called CTAug, to seamlessly integrate cohesion awareness into various existing GCL mechanisms. In particular, CTAug comprises two specialized modules: topology augmentation enhancement and graph learning enhancement. The former module generates augmented graphs that carefully preserve cohesion properties, while the latter module bolsters the graph encoder's ability to discern subgraph patterns. Theoretical analysis shows that CTAug can strictly improve existing GCL mechanisms. Empirical experiments verify that CTAug can achieve state-of-the-art performance for graph representation learning, especially for graphs with high degrees. The code is available at https://doi.org/10.5281/zenodo.10594093, or https://github.com/wuyucheng2002/CTAug.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (64)
  1. Esra Akbas and Peixiang Zhao. 2017. Truss-based Community Search: a Truss-equivalence Based Indexing Approach. Proc. VLDB Endow. 10 (2017), 1298–1309.
  2. Identification of influential spreaders in online social networks using interaction weighted K-core decomposition method. Physica A-statistical Mechanics and Its Applications 468 (2017), 278–288.
  3. Prediction of protein functions based on k-cores of protein-protein interaction networks and amino acid sequences. Genome Informatics 14 (2003), 498–499.
  4. Functional group contributions to drug-receptor interactions. Journal of medicinal chemistry 27, 12 (1984), 1648–1657.
  5. Gary D Bader and Christopher WV Hogue. 2003. An automated method for finding molecular complexes in large protein interaction networks. BMC bioinformatics 4, 1 (2003), 1–27.
  6. Vladimir Batagelj and Matjaz Zaversnik. 2003. An O (m) algorithm for cores decomposition of networks. arXiv preprint cs/0310049 (2003).
  7. Efficient enumeration of maximal k-plexes. In Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data. 431–444.
  8. Protein function prediction via graph kernels. Bioinformatics 21, suppl_1 (2005), i47–i56.
  9. Improving graph neural network expressivity via subgraph isomorphism counting. IEEE Transactions on Pattern Analysis and Machine Intelligence 45, 1 (2022), 657–668.
  10. Phil Brown and Junlan Feng. 2011. Measuring user influence on twitter using modified k-shell decomposition. In Proceedings of the International AAAI Conference on Web and Social Media, Vol. 5. 18–23.
  11. Can graph neural networks count substructures? Advances in neural information processing systems 33 (2020), 10383–10395.
  12. Jonathan Cohen. 2008. Trusses: Cohesive subgraphs for social network analysis. National security agency technical report 16, 3.1 (2008).
  13. A (sub) graph isomorphism algorithm for matching large graphs. IEEE transactions on pattern analysis and machine intelligence 26, 10 (2004), 1367–1372.
  14. Jean-Pierre Eckmann and Elisha Moses. 2002. Curvature of co-links uncovers hidden thematic layers in the World Wide Web. Proceedings of the National Academy of Sciences of the United States of America 99 (2002), 5825 – 5829.
  15. Evaluating cooperation in communities with the k-core structure. In 2011 International conference on advances in social networks analysis and mining. IEEE, 87–93.
  16. Neural message passing for quantum chemistry. In International conference on machine learning. PMLR, 1263–1272.
  17. Kaveh Hassani and Amir Hosein Khasahmadi. 2020. Contrastive multi-view representation learning on graphs. In International Conference on Machine Learning. PMLR, 4116–4126.
  18. Paul Holland and Samuel Leinhardt. 1971. Transitivity in Structural Models of Small Groups. Small Group Research 2 (1971), 107 – 124.
  19. Triadic closure pattern analysis and prediction in social networks. IEEE Transactions on Knowledge and Data Engineering 27, 12 (2015), 3374–3389.
  20. Querying k-truss community in large and dynamic graphs. In Proceedings of the 2014 ACM SIGMOD international conference on Management of data. 1311–1322.
  21. Thomas N Kipf and Max Welling. 2016. Variational graph auto-encoders. arXiv preprint arXiv:1611.07308 (2016).
  22. Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In International Conference on Learning Representations.
  23. Identification of influential spreaders in complex networks. Nature Physics 6 (2010), 888–893.
  24. Diffusion improves graph learning. In Proceedings of the 33rd International Conference on Neural Information Processing Systems. 13366–13378.
  25. k-core: Theories and applications. Physics Reports 832 (2019), 1–32.
  26. Let invariant rationale discovery inspire graph contrastive learning. In International conference on machine learning. PMLR, 13052–13065.
  27. Spectral Augmentation for Self-Supervised Learning on Graphs. In The Eleventh International Conference on Learning Representations.
  28. Revisiting graph contrastive learning from the perspective of graph spectrum. Advances in Neural Information Processing Systems 35 (2022), 2972–2983.
  29. Graph self-supervised learning: A survey. IEEE Transactions on Knowledge and Data Engineering (2022).
  30. The core decomposition of networks: Theory, algorithms and applications. The VLDB Journal 29 (2020), 61–92.
  31. Robert J Mokken et al. 1979. Cliques, clubs and clans. Quality & Quantity 13, 2 (1979), 161–173.
  32. Weisfeiler and leman go neural: Higher-order graph neural networks. In Proceedings of the AAAI conference on artificial intelligence. 4602–4609.
  33. The PageRank Citation Ranking : Bringing Order to the Web. In The Web Conference.
  34. Accelerating Community Detection by Using K-core Subgraphs. ArXiv abs/1403.2226 (2014).
  35. Graph representation learning via graphical mutual information maximization. In Proceedings of The Web Conference 2020. 259–270.
  36. Deepwalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. 701–710.
  37. François Rousseau and Michalis Vazirgiannis. 2015. Main core retention on graph-of-words for single-document keyword extraction. In Advances in Information Retrieval: 37th European Conference on IR Research, ECIR 2015, Vienna, Austria, March 29-April 2, 2015. Proceedings 37. Springer, 382–393.
  38. Karate Club: an API oriented open-source python framework for unsupervised learning on graphs. In Proceedings of the 29th ACM international conference on information & knowledge management. 3125–3132.
  39. Stephen B Seidman. 1983. Network structure and minimum degree. Social networks 5, 3 (1983), 269–287.
  40. Pitfalls of graph neural network evaluation. arXiv preprint arXiv:1811.05868 (2018).
  41. InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization. In International Conference on Learning Representations.
  42. Adversarial graph augmentation to improve graph contrastive learning. Advances in Neural Information Processing Systems 34 (2021), 15920–15933.
  43. What Makes for Good Views for Contrastive Learning?. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (Eds.), Vol. 33. Curran Associates, Inc., 6827–6839.
  44. Augmentations in graph contrastive learning: Current methodological flaws & towards better practices. In Proceedings of the ACM Web Conference 2022. 1538–1549.
  45. Graph attention networks. arXiv preprint arXiv:1710.10903 (2017).
  46. Deep Graph Infomax. ArXiv abs/1809.10341 (2019).
  47. Jia Wang and James Cheng. 2012. Truss decomposition in massive networks. Proceedings of the VLDB Endowment 5, 9 (2012), 812–823.
  48. Heterogeneous graph attention network. In The world wide web conference. 2022–2032.
  49. Multi-view graph contrastive representation learning for drug-drug interaction prediction. In Proceedings of the Web Conference 2021. 2921–2933.
  50. Duncan J Watts and Steven H Strogatz. 1998. Collective dynamics of ‘small-world’networks. nature 393, 6684 (1998), 440–442.
  51. Self-supervised learning on graphs: Contrastive, generative, or predictive. IEEE Transactions on Knowledge and Data Engineering (2021).
  52. Resilience and rewiring of the passenger airline networks in the United States. Physical Review E 82, 5 (2010), 056101.
  53. Simgrace: A simple framework for graph contrastive learning without data augmentation. In Proceedings of the ACM Web Conference 2022. 1070–1079.
  54. How Powerful are Graph Neural Networks?. In International Conference on Learning Representations.
  55. Pinar Yanardag and SVN Vishwanathan. 2015. Deep graph kernels. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining. 1365–1374.
  56. Autogcl: Automated graph contrastive learning via learnable view generators. In Proceedings of the AAAI conference on artificial intelligence, Vol. 36. 8892–8900.
  57. Graph contrastive learning automated. In International Conference on Machine Learning. PMLR, 12121–12132.
  58. Graph contrastive learning with augmentations. Advances in Neural Information Processing Systems 33 (2020), 5812–5823.
  59. Finding critical users for social network engagement: The collapsed k-core problem. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 31.
  60. Motif-Driven Contrastive Learning of Graph Representations. ArXiv abs/2012.12533 (2021).
  61. Hao Zhu and Piotr Koniusz. 2020. Simple spectral graph convolution. In International Conference on Learning Representations.
  62. An Empirical Study of Graph Contrastive Learning. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, Joaquin Vanschoren and Serena Yeung (Eds.), Vol. 1. Curran Associates, Inc.
  63. Deep graph contrastive representation learning. arXiv preprint arXiv:2006.04131 (2020).
  64. Graph contrastive learning with adaptive augmentation. In Proceedings of the Web Conference 2021. 2069–2080.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Github Logo Streamline Icon: https://streamlinehq.com

GitHub

  1. GitHub - wuyucheng2002/CTAug (15 stars)
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets