Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Hypergraph Self-supervised Learning with Sampling-efficient Signals (2404.11825v1)

Published 18 Apr 2024 in cs.LG

Abstract: Self-supervised learning (SSL) provides a promising alternative for representation learning on hypergraphs without costly labels. However, existing hypergraph SSL models are mostly based on contrastive methods with the instance-level discrimination strategy, suffering from two significant limitations: (1) They select negative samples arbitrarily, which is unreliable in deciding similar and dissimilar pairs, causing training bias. (2) They often require a large number of negative samples, resulting in expensive computational costs. To address the above issues, we propose SE-HSSL, a hypergraph SSL framework with three sampling-efficient self-supervised signals. Specifically, we introduce two sampling-free objectives leveraging the canonical correlation analysis as the node-level and group-level self-supervised signals. Additionally, we develop a novel hierarchical membership-level contrast objective motivated by the cascading overlap relationship in hypergraphs, which can further reduce membership sampling bias and improve the efficiency of sample utilization. Through comprehensive experiments on 7 real-world hypergraphs, we demonstrate the superiority of our approach over the state-of-the-art method in terms of both effectiveness and efficiency.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (34)
  1. Deep canonical correlation analysis. In International conference on machine learning, pages 1247–1255. PMLR, 2013.
  2. A survey on hypergraph representation learning. ACM Computing Surveys, 56(1):1–38, 2023.
  3. Hypergraph structure learning for hypergraph neural networks. In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22, pages 1923–1929, 2022.
  4. Scalable and effective deep cca via soft decorrelation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1488–1497, 2018.
  5. You are allset: A multiset function framework for hypergraph neural networks. In International Conference on Learning Representations, 2021.
  6. Hypergraph neural networks. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 3558–3565, 2019.
  7. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9729–9738, 2020.
  8. The total variation on hypergraphs-learning on hypergraphs revisited. Advances in Neural Information Processing Systems, 26, 2013.
  9. On feature decorrelation in self-supervised learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9598–9608, 2021.
  10. Hypergraph convolutional network for group recommendation. In 2021 IEEE International Conference on Data Mining (ICDM), pages 260–269. IEEE, 2021.
  11. Intra-view and inter-view supervised correlation analysis for multi-view feature learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 28, 2014.
  12. Hypergraph attention networks for multimodal learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14581–14590, 2020.
  13. I’m me, we’re us, and i’m us: Tri-directional contrastive learning on hypergraphs. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 8456–8464, 2023.
  14. End-to-end learning of visual representations from uncurated instructional videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9879–9889, 2020.
  15. Bootstrapped representation learning on graphs. In ICLR 2021 Workshop on Geometrical and Topological Representation Learning, 2021.
  16. Deep graph infomax. In International Conference on Learning Representations, 2018.
  17. Augmentations in hypergraph contrastive learning: Fabricated and generative. Advances in neural information processing systems, 35:1909–1922, 2022.
  18. Hypergraph contrastive collaborative filtering. In Proceedings of the 45th International ACM SIGIR conference on research and development in information retrieval, pages 70–79, 2022.
  19. Self-supervised hypergraph transformer for recommender systems. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2100–2109, 2022.
  20. Multi-hypergraph learning-based brain functional connectivity analysis in fmri data. IEEE transactions on medical imaging, 39(5):1746–1758, 2019.
  21. Hypergcn: A new method for training graph convolutional networks on hypergraphs. Advances in neural information processing systems, 32, 2019.
  22. Hyper meta-path contrastive learning for multi-behavior recommendation. In 2021 IEEE International Conference on Data Mining (ICDM), pages 787–796. IEEE, 2021.
  23. Semi-supervised hypergraph node classification on hypergraph line expansion. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, pages 2352–2361, 2022.
  24. Hypergraph convolutional recurrent neural network. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pages 3366–3376, 2020.
  25. Graph contrastive learning with augmentations. Advances in neural information processing systems, 33:5812–5823, 2020.
  26. Adaptive hypergraph learning and its application in image classification. IEEE Transactions on Image Processing, 21(7):3262–3272, 2012.
  27. From canonical correlation analysis to self-supervised graph neural networks. Advances in Neural Information Processing Systems, 34:76–89, 2021.
  28. Double-scale self-supervised hypergraph learning for group recommendation. In Proceedings of the 30th ACM international conference on information & knowledge management, pages 2557–2567, 2021.
  29. Costa: covariance-preserving feature augmentation for graph contrastive learning. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2524–2534, 2022.
  30. Spectral feature augmentation for graph contrastive learning and beyond. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 11289–11297, 2023.
  31. Graph debiased contrastive learning with joint representation clustering. In IJCAI, pages 3434–3440, 2021.
  32. Learning with hypergraphs: Clustering, classification, and embedding. Advances in neural information processing systems, 19, 2006.
  33. Deep graph contrastive representation learning. arXiv preprint arXiv:2006.04131, 2020.
  34. Graph contrastive learning with adaptive augmentation. In Proceedings of the Web Conference 2021, pages 2069–2080, 2021.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now