Higher-Order Networks Representation and Learning: A Survey (2402.19414v2)
Abstract: Network data has become widespread, larger, and more complex over the years. Traditional network data is dyadic, capturing the relations among pairs of entities. With the need to model interactions among more than two entities, significant research has focused on higher-order networks and ways to represent, analyze, and learn from them. There are two main directions to studying higher-order networks. One direction has focused on capturing higher-order patterns in traditional (dyadic) graphs by changing the basic unit of study from nodes to small frequently observed subgraphs, called motifs. As most existing network data comes in the form of pairwise dyadic relationships, studying higher-order structures within such graphs may uncover new insights. The second direction aims to directly model higher-order interactions using new and more complex representations such as simplicial complexes or hypergraphs. Some of these models have long been proposed, but improvements in computational power and the advent of new computational techniques have increased their popularity. Our goal in this paper is to provide a succinct yet comprehensive summary of the advanced higher-order network analysis techniques. We provide a systematic review of its foundations and algorithms, along with use cases and applications of higher-order networks in various scientific domains.
- Network science. Cambridge University Press, Cambridge, 2016. ISBN 9781107076266 1107076269. URL http://barabasi.com/networksciencebook/.
- Exact analytical solutions of the susceptible-infected-recovered (sir) epidemic model and of the sir model with equal death and birth rates. Applied Mathematics and Computation, 236:184–194, 2014. ISSN 0096-3003. doi:https://doi.org/10.1016/j.amc.2014.03.030. URL https://www.sciencedirect.com/science/article/pii/S009630031400383X.
- The pagerank citation ranking: Bringing order to the web. Technical Report 1999-66, Stanford InfoLab, November 1999. URL http://ilpubs.stanford.edu:8090/422/. Previous number = SIDL-WP-1999-0120.
- A survey of shortest-path algorithms. CoRR, abs/1705.02044, 2017. URL http://arxiv.org/abs/1705.02044.
- Heterogeneous network representation learning: Survey, benchmark, evaluation, and beyond. CoRR, abs/2004.00216, 2020. URL https://arxiv.org/abs/2004.00216.
- Community discovery in dynamic networks: A survey. ACM Comput. Surv., 51(2), feb 2018. ISSN 0360-0300. doi:10.1145/3172867. URL https://doi.org/10.1145/3172867.
- Mark Granovetter. The strength of weak ties. The American Journal of Sociology, 78(6):1360–1380, May 1973. URL http://links.jstor.org/sici?sici=0002-9602(197305)78:6%253C1360:TSOWT%253E2.0.CO;2-E.
- Alex Bavelas. Communication Patterns in Task-Oriented Groups. Acoustical Society of America Journal, 22(6):725, January 1950. doi:10.1121/1.1906679.
- S Leinhardt and J Berger. The structure of positive interpersonal relations in small groups. Sociological Theories in Progress. Boston: Houghton Mifflin, 1971.
- Subgraph frequencies: Mapping the empirical and extremal geography of large graph collections. CoRR, abs/1304.1548, 2013. URL http://arxiv.org/abs/1304.1548.
- Network motifs in the transcriptional regulation network of escherichia coli. Nature Genetics, 31:64–68, 2002.
- Network motifs: simple building blocks of complex networks. Science, 298(5594):824–827, October 2002.
- Superfamilies of evolved and designed networks. Science, 303(5663):1538–1542, 2004. doi:10.1126/science.1089167. URL https://www.science.org/doi/abs/10.1126/science.1089167.
- Edwin H. Spanier. Algebraic topology. McGraw-Hill Book, New York, 1966. Includes index.
- Motifs in brain networks. PLoS biology, 2:e369, 12 2004. doi:10.1371/journal.pbio.0020369.
- Brain network motifs are markers of loss and recovery of consciousness. bioRxiv, 2020.
- Spontaneous evolution of modularity and network motifs. Proceedings of the National Academy of Sciences, 102(39):13773–13778, 2005. ISSN 0027-8424. doi:10.1073/pnas.0503610102. URL https://www.pnas.org/content/102/39/13773.
- Signed networks in social media, 2010.
- Social network analysis based on network motifs. Journal of Applied Mathematics, 2014:1–6, 02 2014. doi:10.1155/2014/874708.
- Simplicial closure and higher-order link prediction. CoRR, abs/1802.06916, 2018.
- Higher-order network analysis takes off, fueled by classical ideas and new data. CoRR, abs/2103.05031, 2021. URL https://arxiv.org/abs/2103.05031.
- Multi-Layered Network Embedding, pages 684–692. doi:10.1137/1.9781611975321.77. URL https://epubs.siam.org/doi/abs/10.1137/1.9781611975321.77.
- Multilayer networks. Journal of Complex Networks, 2(3):203–271, 07 2014. ISSN 2051-1310. doi:10.1093/comnet/cnu016. URL https://doi.org/10.1093/comnet/cnu016.
- Towards optimal connectivity on multi-layered networks. IEEE Transactions on Knowledge and Data Engineering, 29(10):2332–2346, 2017. doi:10.1109/TKDE.2017.2719026.
- Understanding complex systems: From networks to optimal higher-order models, 2018.
- Random walks and diffusion on networks. Physics Reports, 716-717:1–58, 2017. ISSN 0370-1573. doi:https://doi.org/10.1016/j.physrep.2017.07.007. URL https://www.sciencedirect.com/science/article/pii/S0370157317302946. Random walks and diffusion on networks.
- Representing higher-order dependencies in networks. Science Advances, 2(5):e1600028, 2016. doi:10.1126/sciadv.1600028. URL https://www.science.org/doi/abs/10.1126/sciadv.1600028.
- Graph signal processing: Overview, challenges, and applications. Proceedings of the IEEE, 106(5):808–828, 2018. doi:10.1109/JPROC.2018.2820126.
- Aliaksei Sandryhaila and José M. F. Moura. Discrete signal processing on graphs. CoRR, abs/1210.4752, 2012. URL http://arxiv.org/abs/1210.4752.
- Signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular data domains. CoRR, abs/1211.0053, 2012. URL http://arxiv.org/abs/1211.0053.
- Big data analysis with signal processing on graphs: Representation and processing of massive data sets with irregular structure. Signal Processing Magazine, IEEE, 31:80–90, 09 2014. doi:10.1109/MSP.2014.2329213.
- Signal processing on higher-order networks: Livin’ on the edge… and beyond. Signal Processing, 187:108149, 2021. ISSN 0165-1684. doi:https://doi.org/10.1016/j.sigpro.2021.108149. URL https://www.sciencedirect.com/science/article/pii/S0165168421001870.
- Complex networks: Structure and dynamics. Physics Reports, 424(4):175–308, 2006. ISSN 0370-1573. doi:https://doi.org/10.1016/j.physrep.2005.10.009. URL https://www.sciencedirect.com/science/article/pii/S037015730500462X.
- What are higher-order networks? CoRR, abs/2104.11329, 2021. URL https://arxiv.org/abs/2104.11329.
- Explosive higher-order kuramoto dynamics on simplicial complexes. Physical Review Letters, 124(21), may 2020. doi:10.1103/physrevlett.124.218301. URL https://doi.org/10.1103%2Fphysrevlett.124.218301.
- Coupled dynamics on hypergraphs: Master stability of steady states and synchronization. Phys. Rev. E, 101:062313, Jun 2020. doi:10.1103/PhysRevE.101.062313. URL https://link.aps.org/doi/10.1103/PhysRevE.101.062313.
- P. Erdös and A. Rényi. On random graphs i. Publicationes Mathematicae Debrecen, 6:290, 1959.
- Higher-order organization of complex networks. Science, 353(6295):163–166, 2016. doi:10.1126/science.aad9029. URL https://www.science.org/doi/abs/10.1126/science.aad9029.
- Collective dynamics of ‘small-world’ networks. Nature, 393(6684):440–442, 1998. doi:10.1038/30918.
- Higher-order clustering in networks. Physical Review E, 97(5), May 2018. ISSN 2470-0053. doi:10.1103/physreve.97.052306. URL http://dx.doi.org/10.1103/PhysRevE.97.052306.
- Cliques and cavities in the human connectome. Journal of Computational Neuroscience, 44:1–31, 02 2018. doi:10.1007/s10827-017-0672-6.
- Modeling interactome: scale-free or geometric? Bioinformatics, 20(18):3508–3515, 07 2004. ISSN 1367-4803. doi:10.1093/bioinformatics/bth436. URL https://doi.org/10.1093/bioinformatics/bth436.
- Local higher-order graph clustering. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’17, page 555–564, New York, NY, USA, 2017. Association for Computing Machinery. ISBN 9781450348874. doi:10.1145/3097983.3098069. URL https://doi.org/10.1145/3097983.3098069.
- Hone: Higher-order network embeddings, 2018.
- Network embedding via motifs. ACM Trans. Knowl. Discov. Data, 16(3), oct 2021. ISSN 1556-4681. doi:10.1145/3473911. URL https://doi.org/10.1145/3473911.
- Link prediction via higher-order motif features. CoRR, abs/1902.06679, 2019. URL http://arxiv.org/abs/1902.06679.
- Control using higher order laplacians in network topologies. In Proc. of 17th International Symposium on Mathematical Theory of Networks and Systems, Kyoto, pages 1024–1038, 2006.
- Lek-Heng Lim. Hodge laplacians on graphs. CoRR, abs/1507.05379, 2015. URL http://arxiv.org/abs/1507.05379.
- Random walks on simplicial complexes and the normalized hodge laplacian. CoRR, abs/1807.05044, 2018. URL http://arxiv.org/abs/1807.05044.
- Generalized network structures: The configuration model and the canonical ensemble of simplicial complexes. Physical Review E, 93(6), jun 2016. doi:10.1103/physreve.93.062311. URL https://doi.org/10.1103%2Fphysreve.93.062311.
- Matthew Kahle. Topology of random simplicial complexes: a survey, 2013. URL https://arxiv.org/abs/1301.7165.
- Coverage and hole-detection in sensor networks via homology. In IPSN ’05: Proceedings of the 4th international symposium on Information processing in sensor networks, Piscataway, NJ, USA, 2005. IEEE Press. ISBN 0-7803-9202-7. URL http://portal.acm.org/citation.cfm?id=1147729.
- Coverage in sensor networks via persistent homology.
- Distributed coverage verification in sensor networks without location information. IEEE Transactions on Automatic Control, 55(8):1837–1849, 2010. doi:10.1109/TAC.2010.2047541.
- Simplicial models and topological inference in biological systems. 2014.
- Disease-specific genomic analysis: identifying the signature of pathologic biology. Bioinformatics, 23(8):957–965, 02 2007. ISSN 1367-4803. doi:10.1093/bioinformatics/btm033. URL https://doi.org/10.1093/bioinformatics/btm033.
- Applications of computational homology to the analysis of treatment response in breast cancer patients. Topology and its Applications, 157:157–164, 2010.
- Topological analysis of gene expression arrays identifies high risk molecular subtypes in breast cancer. Appl. Algebra Eng. Commun. Comput., 23(1-2):3–15, 2012. doi:10.1007/s00200-012-0166-8. URL https://doi.org/10.1007/s00200-012-0166-8.
- Homological scaffolds of brain functional networks. Journal of The Royal Society Interface, 11(101):20140873, 2014. doi:10.1098/rsif.2014.0873. URL https://royalsocietypublishing.org/doi/abs/10.1098/rsif.2014.0873.
- Clique topology reveals intrinsic geometric structure in neural correlations. Proceedings of the National Academy of Sciences, 112(44):13455–13460, 2015. doi:10.1073/pnas.1506407112. URL https://www.pnas.org/doi/abs/10.1073/pnas.1506407112.
- Two’s company, three (or more) is a simplex: Algebraic-topological tools for understanding higher-order structure in neural data, 2016. URL https://arxiv.org/abs/1601.01704.
- Topological signatures for fast mobility analysis. In Proceedings of the 26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, SIGSPATIAL ’18, page 159–168, New York, NY, USA, 2018. Association for Computing Machinery. ISBN 9781450358897. doi:10.1145/3274895.3274952. URL https://doi.org/10.1145/3274895.3274952.
- Construction of and efficient sampling from the simplicial configuration model. Physical Review E, 96(3), sep 2017. doi:10.1103/physreve.96.032312. URL https://doi.org/10.1103%2Fphysreve.96.032312.
- Towards random uniform sampling of bipartite graphs with given degree sequence. 2010. doi:10.48550/ARXIV.1004.2612. URL https://arxiv.org/abs/1004.2612.
- Simplicial models of social contagion. Nature Communications, 10(1), Jun 2019. ISSN 2041-1723. doi:10.1038/s41467-019-10431-6. URL http://dx.doi.org/10.1038/s41467-019-10431-6.
- Emergent hyperbolic network geometry. Scientific Reports, 7(1), feb 2017. doi:10.1038/srep41974. URL https://doi.org/10.1038%2Fsrep41974.
- David F. Gleich. Pagerank beyond the web. CoRR, abs/1407.5107, 2014. URL http://arxiv.org/abs/1407.5107.
- Towards spectral sparsification of simplicial complexes based on generalized effective resistance. CoRR, abs/1708.08436, 2017. URL http://arxiv.org/abs/1708.08436.
- Simplicial complex representation learning. CoRR, abs/2103.04046, 2021. URL https://arxiv.org/abs/2103.04046.
- A survey on hypergraph mining: Patterns, tools, and generators, 2024.
- Tensor decomposition for signal processing and machine learning. IEEE Transactions on Signal Processing, 65(13):3551–3582, 2017.
- Analysis of individual differences in multidimensional scaling via an n-way generalization of “eckart-young” decomposition. Psychometrika, 35:283–319, 1970.
- Richard A. Harshman. Foundations of the parafac procedure: Models and conditions for an "explanatory" multi-model factor analysis. 1970.
- L. R. Tucker. Some mathematical notes on three-mode factor analysis. Psychometrika, 31:279–311, 1966c.
- Hypergraph cuts with general splitting functions. CoRR, abs/2001.02817, 2020a. URL http://arxiv.org/abs/2001.02817.
- High-ordered random walks and generalized laplacians on hypergraphs, 2011.
- Exploiting cross-order patterns and link prediction in higher-order networks. In 2022 IEEE International Conference on Data Mining Workshops (ICDMW), pages 1–9, 2022. doi:10.1109/ICDMW58026.2022.00156.
- Structural patterns and generative models of real-world hypergraphs. pages 176–186, 08 2020. doi:10.1145/3394486.3403060.
- Hypernetwork science via high-order hypergraph walks. EPJ Data Sci., 9(1):16, 2020. doi:10.1140/epjds/s13688-020-00231-0. URL https://doi.org/10.1140/epjds/s13688-020-00231-0.
- Austin R. Benson. Three hypergraph eigenvector centralities. CoRR, abs/1807.09644, 2018. URL http://arxiv.org/abs/1807.09644.
- Higher-order motif analysis in hypergraphs. Communications Physics, 5(1), April 2022. ISSN 2399-3650. doi:10.1038/s42005-022-00858-7. URL http://dx.doi.org/10.1038/s42005-022-00858-7.
- Hypergraph motifs: Concepts, algorithms, and discoveries. CoRR, abs/2003.01853, 2020. URL https://arxiv.org/abs/2003.01853.
- Philip S Chodrow. Configuration models of random hypergraphs. Journal of Complex Networks, 8(3), 08 2020. ISSN 2051-1329. doi:10.1093/comnet/cnaa018. URL https://doi.org/10.1093/comnet/cnaa018. cnaa018.
- How do hyperedges overlap in real-world hypergraphs? - patterns, measures, and generators. CoRR, abs/2101.07480, 2021. URL https://arxiv.org/abs/2101.07480.
- The average distances in random graphs with given expected degrees. Proceedings of the National Academy of Sciences, 99(25):15879–15882, 2002. doi:10.1073/pnas.252631999. URL https://www.pnas.org/doi/abs/10.1073/pnas.252631999.
- Stochastic blockmodels and community structure in networks. Physical Review E, 83(1), Jan 2011. ISSN 1550-2376. doi:10.1103/physreve.83.016107. URL http://dx.doi.org/10.1103/PhysRevE.83.016107.
- Generative hypergraph clustering: From blockmodels to modularity. Science Advances, 7(28):eabh1303, 2021. doi:10.1126/sciadv.abh1303. URL https://www.science.org/doi/abs/10.1126/sciadv.abh1303.
- Fast unfolding of communities in large networks. Journal of statistical mechanics: theory and experiment, 2008(10):P10008, 2008.
- Efficient sampling algorithm for estimating subgraph concentrations and detecting network motifs. Bioinformatics (Oxford, England), 20:1746–58, 08 2004. doi:10.1093/bioinformatics/bth163.
- Frequency concepts and pattern detection for the analysis of motifs in networks. In Corrado Priami, Emanuela Merelli, Pablo Gonzalez, and Andrea Omicini, editors, Transactions on Computational Systems Biology III, pages 89–104, Berlin, Heidelberg, 2005. Springer Berlin Heidelberg. ISBN 978-3-540-31446-2.
- Sebastian Wernicke. Efficient detection of network motifs. IEEE/ACM Trans. Comput. Biol. Bioinformatics, 3(4):347–359, oct 2006. ISSN 1545-5963. doi:10.1109/TCBB.2006.51. URL https://doi.org/10.1109/TCBB.2006.51.
- Network motif discovery using subgraph enumeration and symmetry-breaking. In Proceedings of the 11th Annual International Conference on Research in Computational Molecular Biology, RECOMB’07, page 92–106, Berlin, Heidelberg, 2007. Springer-Verlag. ISBN 9783540716808.
- Moda: An efficient algorithm for network motif discovery in biological networks. Genes & genetic systems, 84:385–95, 10 2009. doi:10.1266/ggs.84.385.
- Kavosh : a new algorithm for finding network motifs. BMC Bioinformatics, 10, 2009. doi:10.1186/1471-2105-10-318. Article Number: 318.
- G-tries: An efficient data structure for discovering network motifs. pages 1559–1566, 01 2010. doi:10.1145/1774088.1774422.
- Motifs in temporal networks. CoRR, abs/1612.09259, 2016. URL http://arxiv.org/abs/1612.09259.
- Application of dynamic expansion tree for finding large network motifs in biological networks. PeerJ, 7:e6917, 05 2019. doi:10.7717/peerj.6917.
- JavaPlex: A research software package for persistent (co)homology. In Han Hong and Chee Yap, editors, Proceedings of ICMS 2014, Lecture Notes in Computer Science 8592, pages 129–136, 2014. Software available at http://appliedtopology.github.io/javaplex/.
- Ulrich Bauer. Ripser: efficient computation of vietoris-rips persistence barcodes. Journal of Applied and Computational Topology, 2021. doi:10.1007/s41468-021-00071-5.
- Morse theory for filtrations and efficient computation of persistent homology. Discrete Comput. Geom., 50(2):330–353, sep 2013. ISSN 0179-5376. doi:10.1007/s00454-013-9529-6. URL https://doi.org/10.1007/s00454-013-9529-6.
- Topological deep learning: Going beyond graph data, 2023.
- Minimizing Localized Ratio Cut Objectives in Hypergraphs, page 1708–1718. Association for Computing Machinery, New York, NY, USA, 2020b. ISBN 9781450379984. URL https://doi.org/10.1145/3394486.3403222.
- Tensor spectral clustering for partitioning higher-order network structures. CoRR, abs/1502.05058, 2015.
- Clustering in Graphs and Hypergraphs with Categorical Edge Labels, page 706–717. Association for Computing Machinery, New York, NY, USA, 2020. ISBN 9781450370233. URL https://doi.org/10.1145/3366423.3380152.
- Strongly local hypergraph diffusions for clustering and semi-supervised learning. Proceedings of the Web Conference 2021. doi:10.1145/3442381.3449887. URL https://par.nsf.gov/biblio/10285904.
- Using rich social media information for music recommendation via hypergraph model. ACM Trans. Multimedia Comput. Commun. Appl., 7S(1), nov 2011. ISSN 1551-6857. doi:10.1145/2037676.2037679. URL https://doi.org/10.1145/2037676.2037679.
- Hypergraph with sampling for image retrieval. Pattern Recognition, 44(10):2255–2262, 2011. ISSN 0031-3203. doi:https://doi.org/10.1016/j.patcog.2010.07.014. URL https://www.sciencedirect.com/science/article/pii/S0031320310003535. Semi-Supervised Learning for Visual Content Analysis and Understanding.
- Predicting protein interactions via parsimonious network history inference. Bioinformatics, 29(13):i237–i246, 06 2013. ISSN 1367-4803. doi:10.1093/bioinformatics/btt224. URL https://doi.org/10.1093/bioinformatics/btt224.
- Hypergraph neural networks, 2018. URL https://arxiv.org/abs/1809.09401.
- Hypergraph convolution and hypergraph attention. CoRR, abs/1901.08150, 2019. URL http://arxiv.org/abs/1901.08150.
- Session-based recommendation with hypergraph attention networks. CoRR, abs/2112.14266, 2021. URL https://arxiv.org/abs/2112.14266.
- Self-supervised multi-channel hypergraph convolutional network for social recommendation. CoRR, abs/2101.06448, 2021. URL https://arxiv.org/abs/2101.06448.
- Hypergcn: A new method of training graph convolutional networks on hypergraphs, 2018. URL https://arxiv.org/abs/1809.02589.
- Next-Item Recommendation with Sequential Hypergraphs, page 1101–1110. Association for Computing Machinery, New York, NY, USA, 2020. ISBN 9781450380164. URL https://doi.org/10.1145/3397271.3401133.
- Graph neural networks designed for different graph types: A survey, 04 2022.
- Hypernetwork science: From multidimensional networks to computational topology. CoRR, abs/2003.11782, 2020. URL https://arxiv.org/abs/2003.11782.
- Graph markup language (graphml). In Handbook of Graph Drawing and Visualization, 2013.
- Analyzing, exploring, and visualizing complex networks via hypergraphs using SimpleHypergraphs.jl. Internet Mathematics, apr 2020. doi:10.24166/im.01.2020. URL https://doi.org/10.24166%2Fim.01.2020.
- Sebastian Schlag. High-Quality Hypergraph Partitioning. PhD thesis, Karlsruhe Institute of Technology, Germany, 2020.
- Tensorly: Tensor learning in python. Journal of Machine Learning Research, 20(26):1–6, 2019. URL http://jmlr.org/papers/v20/18-277.html.
- Tensors.jl — tensor computations in julia. Journal of Open Research Software, 7, 03 2019. doi:10.5334/jors.182.
- rtensor: An r package for multidimensional array (tensor) unfolding, multiplication, and decomposition. Journal of Statistical Software, 87(10):1–31, 2018. doi:10.18637/jss.v087.i10. URL https://www.jstatsoft.org/index.php/jss/article/view/v087i10.
- Hypergraph link prediction: Learning drug interaction networks embeddings. In 2019 18th IEEE International Conference On Machine Learning And Applications (ICMLA), pages 1860–1865, 2019. doi:10.1109/ICMLA.2019.00299.
- Generative hypergraph models and spectral embedding, 2023.
- Datasets, tasks, and training methods for large-scale hypergraph learning. Data Mining and Knowledge Discovery, 37:1–39, 07 2023. doi:10.1007/s10618-023-00952-6.
- Efficient graphlet counting for large networks. In 2015 IEEE International Conference on Data Mining, pages 1–10, 2015. doi:10.1109/ICDM.2015.141.
- D. Marcus and Y. Shavitt. Rage – a rapid graphlet enumerator for large networks. Computer Networks, 56(2):810–819, 2012. ISSN 1389-1286. doi:https://doi.org/10.1016/j.comnet.2011.08.019. URL https://www.sciencedirect.com/science/article/pii/S1389128611003902.
- Zigzag persistent homology in matrix multiplication time. In Proceedings of the Twenty-Seventh Annual Symposium on Computational Geometry, SoCG ’11, page 216–225, New York, NY, USA, 2011. Association for Computing Machinery. ISBN 9781450306829. doi:10.1145/1998196.1998229. URL https://doi.org/10.1145/1998196.1998229.
- Group recommendation: semantics and efficiency. Proc. VLDB Endow., 2(1):754–765, aug 2009. ISSN 2150-8097. doi:10.14778/1687627.1687713. URL https://doi.org/10.14778/1687627.1687713.