Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
36 tokens/sec
GPT-4o
12 tokens/sec
Gemini 2.5 Pro Pro
37 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
4 tokens/sec
DeepSeek R1 via Azure Pro
33 tokens/sec
2000 character limit reached

Unifying Generation and Prediction on Graphs with Latent Graph Diffusion (2402.02518v2)

Published 4 Feb 2024 in cs.LG and cs.SI

Abstract: In this paper, we propose the first framework that enables solving graph learning tasks of all levels (node, edge and graph) and all types (generation, regression and classification) using one formulation. We first formulate prediction tasks including regression and classification into a generic (conditional) generation framework, which enables diffusion models to perform deterministic tasks with provable guarantees. We then propose Latent Graph Diffusion (LGD), a generative model that can generate node, edge, and graph-level features of all categories simultaneously. We achieve this goal by embedding the graph structures and features into a latent space leveraging a powerful encoder and decoder, then training a diffusion model in the latent space. LGD is also capable of conditional generation through a specifically designed cross-attention mechanism. Leveraging LGD and the ``all tasks as generation'' formulation, our framework is capable of solving graph tasks of various levels and types. We verify the effectiveness of our framework with extensive experiments, where our models achieve state-of-the-art or highly competitive results across a wide range of generation and regression tasks.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (52)
  1. Error bounds for flow matching methods. ArXiv, abs/2305.16860, 2023. URL https://api.semanticscholar.org/CorpusID:258947044.
  2. Weisfeiler and lehman go cellular: Cw networks. In Neural Information Processing Systems, 2021.
  3. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258, 2021.
  4. Exploring the optimal choice for generative processes in diffusion models: Ordinary vs stochastic differential equations. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openreview.net/forum?id=1mJQq6zYaE.
  5. Score approximation, estimation and distribution recovery of diffusion models on low-dimensional data. In Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., and Scarlett, J. (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp.  4672–4712. PMLR, 23–29 Jul 2023. URL https://proceedings.mlr.press/v202/chen23o.html.
  6. Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions. arXiv preprint arXiv:2209.11215, 2022.
  7. Can graph neural networks count substructures? CoRR, abs/2002.04025, 2020. URL https://arxiv.org/abs/2002.04025.
  8. Principal neighbourhood aggregation for graph nets. ArXiv, abs/2004.05718, 2020.
  9. Re-think and re-design graph neural networks in spaces of continuous graph diffusion functionals. 2023.
  10. Closing the ode-sde gap in score-based diffusion models through the fokker-planck equation. ArXiv, abs/2311.15996, 2023. URL https://api.semanticscholar.org/CorpusID:265456014.
  11. Benchmarking graph neural networks. ArXiv, abs/2003.00982, 2020.
  12. How powerful are k-hop message passing graph neural networks. ArXiv, abs/2205.13328, 2022.
  13. Understanding and extending subgraph gnns by rethinking their symmetries. ArXiv, abs/2206.11140, 2022.
  14. Graphite: Iterative generative modeling of graphs. In International conference on machine learning, pp. 2434–2444. PMLR, 2019.
  15. Diffusion models for graphs benefit from discrete state spaces. ArXiv, abs/2210.01549, 2022.
  16. Denoising diffusion probabilistic models. ArXiv, abs/2006.11239, 2020.
  17. Equivariant diffusion for molecule generation in 3d. ArXiv, abs/2203.17003, 2022. URL https://api.semanticscholar.org/CorpusID:247839510.
  18. Strategies for pre-training graph neural networks. arXiv: Learning, 2019.
  19. Open graph benchmark: Datasets for machine learning on graphs. ArXiv, abs/2005.00687, 2020.
  20. Boosting the Cycle Counting Power of Graph Neural Networks with I22{}^{2}start_FLOATSUPERSCRIPT 2 end_FLOATSUPERSCRIPT-GNNs. arXiv e-prints, art. arXiv:2210.13978, October 2022. doi: 10.48550/arXiv.2210.13978.
  21. Global self-attention as a replacement for graph convolution. Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2021. URL https://api.semanticscholar.org/CorpusID:249375304.
  22. Score-based generative modeling of graphs via the system of stochastic differential equations. In International Conference on Machine Learning, 2022.
  23. Auto-encoding variational bayes. CoRR, abs/1312.6114, 2013.
  24. Convergence of score-based generative modeling for general data distributions. ArXiv, abs/2209.12381, 2022. URL https://api.semanticscholar.org/CorpusID:252531877.
  25. Your diffusion model is secretly a zero-shot classifier. ArXiv, abs/2303.16203, 2023a. URL https://api.semanticscholar.org/CorpusID:257771787.
  26. On the generalization properties of diffusion models. In Thirty-seventh Conference on Neural Information Processing Systems, 2023b. URL https://openreview.net/forum?id=hCUG1MCFk5.
  27. Multi-objective de novo drug design with conditional graph generative model. Journal of cheminformatics, 10:1–24, 2018.
  28. Sagess: Sampling graph denoising diffusion model for scalable graph generation. ArXiv, abs/2306.16827, 2023.
  29. Flow straight and fast: Learning to generate and transfer data with rectified flow. ArXiv, abs/2209.03003, 2022. URL https://api.semanticscholar.org/CorpusID:252111177.
  30. Graph inductive biases in transformers without message passing. ArXiv, abs/2305.17589, 2023.
  31. Graphnvp: An invertible flow model for generating molecular graphs. arXiv preprint arXiv:1905.11600, 2019.
  32. SPECTRE: Spectral conditioning helps to overcome the expressivity limits of one-shot graph generators. In Chaudhuri, K., Jegelka, S., Song, L., Szepesvari, C., Niu, G., and Sabato, S. (eds.), Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp.  15159–15179. PMLR, 17–23 Jul 2022. URL https://proceedings.mlr.press/v162/martinkus22a.html.
  33. Permutation invariant graph generation via score-based generative modeling. In International Conference on Artificial Intelligence and Statistics, 2020.
  34. Ordered subgraph aggregation networks. ArXiv, abs/2206.11168, 2022.
  35. Quantum chemistry structures and properties of 134 kilo molecules. Scientific Data, 1, 2014.
  36. Recipe for a general, powerful, scalable graph transformer. ArXiv, abs/2205.12454, 2022.
  37. High-resolution image synthesis with latent diffusion models. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.  10674–10685, 2021.
  38. Denoising diffusion implicit models. ArXiv, abs/2010.02502, 2020a.
  39. Generative modeling by estimating gradients of the data distribution. ArXiv, abs/1907.05600, 2019.
  40. Improved techniques for training score-based generative models. ArXiv, abs/2006.09011, 2020.
  41. Score-based generative modeling through stochastic differential equations. ArXiv, abs/2011.13456, 2020b.
  42. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023.
  43. Neural discrete representation learning. ArXiv, abs/1711.00937, 2017.
  44. Top-n: Equivariant set and graph generation without exchangeability. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=-Gk_IPJWvk.
  45. Digress: Discrete denoising diffusion for graph generation. ArXiv, abs/2209.14734, 2022.
  46. How powerful are graph neural networks? ArXiv, abs/1810.00826, 2018.
  47. Geometric latent diffusion models for 3d molecule generation. In International Conference on Machine Learning, 2023. URL https://api.semanticscholar.org/CorpusID:258436871.
  48. Vqgraph: Graph vector-quantization for bridging gnns and mlps. ArXiv, 2023a.
  49. Directional diffusion models for graph representation learning. ArXiv, abs/2306.13210, 2023b.
  50. Reward-directed conditional diffusion: Provable distribution estimation and reward improvement. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openreview.net/forum?id=58HwnnEdtF.
  51. Nested graph neural networks. ArXiv, abs/2110.13197, 2021.
  52. From stars to subgraphs: Uplifting any gnn with local structure awareness. ArXiv, abs/2110.03753, 2021.
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Follow-up Questions

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now