Control-based Graph Embeddings with Data Augmentation for Contrastive Learning
Abstract: In this paper, we study the problem of unsupervised graph representation learning by harnessing the control properties of dynamical networks defined on graphs. Our approach introduces a novel framework for contrastive learning, a widely prevalent technique for unsupervised representation learning. A crucial step in contrastive learning is the creation of 'augmented' graphs from the input graphs. Though different from the original graphs, these augmented graphs retain the original graph's structural characteristics. Here, we propose a unique method for generating these augmented graphs by leveraging the control properties of networks. The core concept revolves around perturbing the original graph to create a new one while preserving the controllability properties specific to networks and graphs. Compared to the existing methods, we demonstrate that this innovative approach enhances the effectiveness of contrastive learning frameworks, leading to superior results regarding the accuracy of the classification tasks. The key innovation lies in our ability to decode the network structure using these control properties, opening new avenues for unsupervised graph representation learning.
- W. Hamilton, Z. Ying, and J. Leskovec, “Inductive representation learning on large graphs,” Advances in neural information processing systems, vol. 30, 2017.
- A. v. d. Oord, Y. Li, and O. Vinyals, “Representation learning with contrastive predictive coding,” arXiv preprint arXiv:1807.03748, 2018.
- T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” in International conference on machine learning. PMLR, 2020, pp. 1597–1607.
- Y. You, T. Chen, Y. Sui, T. Chen, Z. Wang, and Y. Shen, “Graph contrastive learning with augmentations,” Advances in neural information processing systems, vol. 33, pp. 5812–5823, 2020.
- L. Xia, C. Huang, Y. Xu, J. Zhao, D. Yin, and J. Huang, “Hypergraph contrastive collaborative filtering,” in Proceedings of the 45th International ACM SIGIR conference on research and development in information retrieval, 2022, pp. 70–79.
- A. Yazıcıoğlu, W. Abbas, and M. Egerstedt, “Graph distances and controllability of networks,” IEEE Transactions on Automatic Control, vol. 61, no. 12, pp. 4125–4130, 2016.
- N. Monshizadeh, S. Zhang, and M. K. Camlibel, “Zero forcing sets and controllability of dynamical systems defined on graphs,” IEEE Transactions on Automatic Control, vol. 59, pp. 2562–2567, 2014.
- H. Duan, C. Xie, B. Li, and P. Tang, “Self-supervised contrastive graph representation with node and graph augmentation,” Neural Networks, 2023.
- F.-Y. Sun, J. Hoffmann, V. Verma, and J. Tang, “Infograph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization,” arXiv preprint arXiv:1908.01000, 2019.
- K. Hassani and A. H. Khasahmadi, “Contrastive multi-view representation learning on graphs,” in International conference on machine learning. PMLR, 2020, pp. 4116–4126.
- F. Pasqualetti, S. Zampieri, and F. Bullo, “Controllability metrics, limitations and algorithms for complex networks,” IEEE Transactions on Control of Network Systems, vol. 1, no. 1, pp. 40–52, 2014.
- T. H. Summers, F. L. Cortesi, and J. Lygeros, “On submodularity and controllability in complex dynamical networks,” IEEE Transactions on Control of Network Systems, vol. 3, no. 1, pp. 91–101, 2015.
- E. Wu-Yan, R. F. Betzel, E. Tang, S. Gu, F. Pasqualetti, and D. S. Bassett, “Benchmarking measures of network controllability on canonical graph models,” Journal of Nonlinear Science, pp. 1–39, 2018.
- A. Chapman and M. Mesbahi, “On strong structural controllability of networked systems: A constrained matching approach,” in American Control Conference, 2013, pp. 6126–6131.
- N. Monshizadeh, K. Camlibel, and H. Trentelman, “Strong targeted controllability of dynamical networks,” in 2015 54th IEEE Conference on Decision and Control (CDC), 2015, pp. 4782–4787.
- O. U. Ahmad, W. Abbas, and M. Shabbir, “Controllability backbone in networks,” to appear in CDC, 2023, arXiv:2309.02649.
- W. Abbas, M. Shabbir, H. Jaleel, and X. Koutsoukos, “Improving network robustness through edge augmentation while preserving strong structural controllability,” in 2020 American Control Conference (ACC). IEEE, 2020, pp. 2544–2549.
- C. Morris, N. M. Kriege, F. Bause, K. Kersting, P. Mutzel, and M. Neumann, “Tudataset: A collection of benchmark datasets for learning with graphs,” in ICML 2020 Workshop on Graph Representation Learning and Beyond, 2020.
- A. Narayanan, M. Chandramohan, R. Venkatesan, L. Chen, Y. Liu, and S. Jaiswal, “graph2vec: Learning distributed representations of graphs,” arXiv preprint arXiv:1707.05005, 2017.
- N. Shervashidze, S. Vishwanathan, T. Petri, K. Mehlhorn, and K. Borgwardt, “Efficient graphlet kernels for large graph comparison,” in Artificial intelligence and statistics. PMLR, 2009, pp. 488–495.
- N. Shervashidze, P. Schweitzer, E. J. Van Leeuwen, K. Mehlhorn, and K. M. Borgwardt, “Weisfeiler-lehman graph kernels.” Journal of Machine Learning Research, vol. 12, no. 9, 2011.
- P. Yanardag and S. Vishwanathan, “Deep graph kernels,” in Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, 2015, pp. 1365–1374.
- A. Grover and J. Leskovec, “node2vec: Scalable feature learning for networks,” in Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016.
- B. Adhikari, Y. Zhang, N. Ramakrishnan, and B. A. Prakash, “Sub2vec: Feature learning for subgraphs,” in Advances in Knowledge Discovery and Data Mining. Springer, 2018, pp. 170–182.
Paper Prompts
Sign up for free to create and run prompts on this paper.