Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

HiGen: Hierarchical Graph Generative Networks (2305.19337v2)

Published 30 May 2023 in cs.LG and cs.SI

Abstract: Most real-world graphs exhibit a hierarchical structure, which is often overlooked by existing graph generation methods. To address this limitation, we propose a novel graph generative network that captures the hierarchical nature of graphs and successively generates the graph sub-structures in a coarse-to-fine fashion. At each level of hierarchy, this model generates communities in parallel, followed by the prediction of cross-edges between communities using separate neural networks. This modular approach enables scalable graph generation for large and complex graphs. Moreover, we model the output distribution of edges in the hierarchical graph with a multinomial distribution and derive a recursive factorization for this distribution. This enables us to generate community graphs with integer-valued edge weights in an autoregressive manner. Empirical studies demonstrate the effectiveness and scalability of our proposed generative model, achieving state-of-the-art performance in terms of graph quality across various benchmark datasets. The code is available at https://github.com/Karami-m/HiGen_main.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (55)
  1. Clustering with bregman divergences. Journal of machine learning research, 6(10), 2005.
  2. Emergence of scaling in random networks. science, 286(5439):509–512, 1999.
  3. Latent dirichlet allocation. Journal of machine Learning research, 3(Jan):993–1022, 2003.
  4. Fast unfolding of communities in large networks. Journal of statistical mechanics: theory and experiment, 2008(10):P10008, 2008.
  5. Netgan: Generating graphs via random walks. arXiv preprint arXiv:1803.00816, 2018.
  6. On the power of edge independent graph models. Advances in Neural Information Processing Systems, 34:24418–24429, 2021.
  7. Order matters: Probabilistic modeling of node sequence for graph generation. arXiv preprint arXiv:2106.06189, 2021.
  8. Efficient and degree-guided graph generation via discrete diffusion modeling. arXiv preprint arXiv:2305.04111, 2023.
  9. Rethinking attention with performers. arXiv preprint arXiv:2009.14794, 2020.
  10. Scalable deep generative modeling for sparse graphs. In International Conference on Machine Learning, pp. 2302–2312. PMLR, 2020.
  11. Molgan: An implicit generative model for small molecular graphs. arXiv preprint arXiv:1805.11973, 2018.
  12. Distinguishing enzyme structures from non-enzymes without alignments. Journal of molecular biology, 330(4):771–783, 2003.
  13. A generalization of transformer networks to graphs. arXiv preprint arXiv:2012.09699, 2020.
  14. On the evolution of random graphs. Publ. Math. Inst. Hung. Acad. Sci, 5(1):17–60, 1960.
  15. Generative adversarial networks. Communications of the ACM, 63(11):139–144, 2020.
  16. Diffusion models for graphs benefit from discrete state spaces. arXiv preprint arXiv:2210.01549, 2022.
  17. Visual recognition by counting instances: A multi-instance cardinality potential kernel. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.  2596–2605, 2015.
  18. Hierarchical generation of molecular graphs using structural motifs. In International conference on machine learning, pp. 4839–4848. PMLR, 2020.
  19. Score-based generative modeling of graphs via the system of stochastic differential equations. In International Conference on Machine Learning, pp. 10362–10383. PMLR, 2022.
  20. Invertible convolutional flow. Advances in Neural Information Processing Systems, 32, 2019.
  21. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  22. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.
  23. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016.
  24. Autoregressive diffusion model for graph generation, 2023.
  25. Molgrow: A graph normalizing flow for hierarchical molecular generation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp.  8226–8234, 2021.
  26. Kronecker graphs: An approach to modeling networks. Journal of Machine Learning Research, 11(Feb):985–1042, 2010.
  27. Learning deep generative models of graphs. arXiv preprint arXiv:1803.03324, 2018.
  28. Efficient graph generation with graph recurrent attention networks. Advances in neural information processing systems, 32, 2019.
  29. Dependent multinomial models made easy: Stick-breaking with the pólya-gamma augmentation. Advances in Neural Information Processing Systems, 28, 2015.
  30. Graph normalizing flows, 2019.
  31. Constrained generation of semantically valid graphs via regularizing variational autoencoders. arXiv preprint arXiv:1809.02630, 2018.
  32. Habitat: A Platform for Embodied AI Research. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019.
  33. Spectre: Spectral conditioning helps to overcome the expressivity limits of one-shot graph generators. In International Conference on Machine Learning, pp. 15159–15179. PMLR, 2022.
  34. A graph placement methodology for fast chip design. Nature, 594(7862):207–212, 2021.
  35. Graph kernels for object category prediction in task-dependent robot grasping. In International Workshop on Mining and Learning with Graphs at KDD, 2013.
  36. Mark EJ Newman. Finding community structure in networks using the eigenvectors of matrices. Physical review E, 74(3):036104, 2006a.
  37. Mark EJ Newman. Modularity and community structure in networks. Proceedings of the national academy of sciences, 103(23):8577–8582, 2006b.
  38. Evaluation metrics for graph generative models: Problems, pitfalls, and practical solutions. arXiv preprint arXiv:2106.01098, 2021.
  39. Wavenet: A generative model for raw audio. arXiv preprint arXiv:1609.03499, 2016.
  40. Habitat-matterport 3d dataset (HM3d): 1000 large-scale 3d environments for embodied AI. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021.
  41. Recipe for a general, powerful, scalable graph transformer. Advances in Neural Information Processing Systems, 35:14501–14515, 2022.
  42. Parallel multiscale autoregressive density estimation. In International Conference on Machine Learning, pp. 2912–2921. PMLR, 2017.
  43. Brenda, the enzyme database: updates and major new developments. Nucleic acids research, 32(suppl_1):D431–D433, 2004.
  44. Collective classification in network data. AI magazine, 29(3):93–93, 2008.
  45. Normalized cuts and image segmentation. IEEE Transactions on pattern analysis and machine intelligence, 22(8):888–905, 2000.
  46. Td-gen: Graph generation using tree decomposition. In International Conference on Artificial Intelligence and Statistics, pp.  5518–5537. PMLR, 2022.
  47. Exphormer: Sparse transformers for graphs. arXiv preprint arXiv:2303.06147, 2023.
  48. Kyle Siegrist. Probability, Mathematical Statistics, Stochastic Processes. LibreTexts, 2017. URL https://stats.libretexts.org/Bookshelves/Probability_Theory/Probability_Mathematical_Statistics_and_Stochastic_Processes_(Siegrist).
  49. GraphVAE: Towards generation of small graphs using variational autoencoders. arXiv preprint arXiv:1802.03480, 2018.
  50. On evaluation metrics for graph generative models. arXiv preprint arXiv:2201.09871, 2022.
  51. Graph clustering with graph neural networks. arXiv preprint arXiv:2006.16904, 2020.
  52. Digress: Discrete denoising diffusion for graph generation. arXiv preprint arXiv:2209.14734, 2022.
  53. Order matters: Sequence to sequence for sets. arXiv preprint arXiv:1511.06391, 2015.
  54. Conditional structure generation through graph variational generative adversarial nets. Advances in neural information processing systems, 32, 2019.
  55. Graphrnn: Generating realistic graphs with deep auto-regressive models. In ICML, pp.  5694–5703, 2018.
Citations (3)

Summary

We haven't generated a summary for this paper yet.