Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
175 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Graph Contrastive Invariant Learning from the Causal Perspective (2401.12564v2)

Published 23 Jan 2024 in cs.LG and cs.SI

Abstract: Graph contrastive learning (GCL), learning the node representation by contrasting two augmented graphs in a self-supervised way, has attracted considerable attention. GCL is usually believed to learn the invariant representation. However, does this understanding always hold in practice? In this paper, we first study GCL from the perspective of causality. By analyzing GCL with the structural causal model (SCM), we discover that traditional GCL may not well learn the invariant representations due to the non-causal information contained in the graph. How can we fix it and encourage the current GCL to learn better invariant representations? The SCM offers two requirements and motives us to propose a novel GCL method. Particularly, we introduce the spectral graph augmentation to simulate the intervention upon non-causal factors. Then we design the invariance objective and independence objective to better capture the causal factors. Specifically, (i) the invariance objective encourages the encoder to capture the invariant information contained in causal variables, and (ii) the independence objective aims to reduce the influence of confounders on the causal variables. Experimental results demonstrate the effectiveness of our approach on node classification tasks.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (37)
  1. Invariant risk minimization. arXiv preprint arXiv:1907.02893.
  2. Spectral networks and locally connected networks on graphs. In ICLR.
  3. A simple framework for contrastive learning of visual representations. In ICML, 1597–1607.
  4. Learning causally invariant representations for out-of-distribution generalization on graphs. Advances in Neural Information Processing Systems, 35: 22131–22148.
  5. Pearson correlation coefficient. Noise reduction in speech processing, 1–4.
  6. Convolutional neural networks on graphs with fast localized spectral filtering. NeurIPS.
  7. Debiasing Graph Neural Networks via Learning Disentangled Causal Substructure. In NeurIPS.
  8. Generalizing graph neural networks on out-of-distribution graphs. IEEE Transactions on Pattern Analysis and Machine Intelligence.
  9. Debiased graph neural networks with agnostic label selection bias. IEEE transactions on neural networks and learning systems.
  10. Directed acyclic graph structure learning from dynamic graphs. In Proceedings of the AAAI Conference on Artificial Intelligence, 7512–7521.
  11. Inductive representation learning on large graphs. NeurIPS.
  12. Contrastive multi-view representation learning on graphs. In ICML, 4116–4126.
  13. Momentum contrast for unsupervised visual representation learning. In CVPR, 9729–9738.
  14. Semi-Supervised Classification with Graph Convolutional Networks. In ICLR.
  15. Let invariant rationale discovery inspire graph contrastive learning. In ICML, 13052–13065.
  16. Revisiting Graph Contrastive Learning from the Perspective of Graph Spectrum. In NeurIPS.
  17. Graph self-supervised learning: A survey. IEEE Transactions on Knowledge and Data Engineering.
  18. Wiki-cs: A wikipedia-based benchmark for graph neural networks. arXiv preprint arXiv:2007.02901.
  19. Variance covariance regularization enforces pairwise independence in self-supervised representations. arXiv preprint arXiv:2209.14905.
  20. Representation Learning via Invariant Causal Mechanisms. In ICLR.
  21. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748.
  22. Pearl, J. 2009. Causality. Cambridge university press.
  23. Causal inference in statistics: A primer. John Wiley & Sons.
  24. Gcc: Graph contrastive coding for graph neural network pre-training. In KDD, 1150–1160.
  25. Graph Attention Networks. In ICLR.
  26. Deep Graph Infomax. ICLR.
  27. Discovering Invariant Rationales for Graph Neural Networks. In ICLR.
  28. A comprehensive survey on graph neural networks. IEEE transactions on neural networks and learning systems, 4–24.
  29. Progcl: Rethinking hard negative mining in graph contrastive learning. In ICML, 24332–24346. PMLR.
  30. Self-supervised learning of graph neural networks: A unified review. IEEE Transactions on Pattern Analysis and Machine Intelligence.
  31. Graph contrastive learning with augmentations. NeurIPS, 5812–5823.
  32. Causal intervention for weakly-supervised semantic segmentation. NeurIPS, 655–666.
  33. From canonical correlation analysis to self-supervised graph neural networks. NeurIPS, 76–89.
  34. COSTA: Covariance-Preserving Feature Augmentation for Graph Contrastive Learning. In KDD, 2524–2534.
  35. Graph neural networks: A review of methods and applications. AI Open, 57–81.
  36. Deep graph contrastive representation learning. arXiv preprint arXiv:2006.04131.
  37. Graph contrastive learning with adaptive augmentation. In TheWebConf, 2069–2080.
Citations (7)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets