Complex Preferences for Different Convergent Priors in Discrete Graph Diffusion
Abstract: Diffusion models have achieved state-of-the-art performance in generating many different kinds of data, including images, text, and videos. Despite their success, there has been limited research on how the underlying diffusion process and the final convergent prior can affect generative performance; this research has also been limited to continuous data types and a score-based diffusion framework. To fill this gap, we explore how different discrete diffusion kernels (which converge to different prior distributions) affect the performance of diffusion models for graphs. To this end, we developed a novel formulation of a family of discrete diffusion kernels which are easily adjustable to converge to different Bernoulli priors, and we study the effect of these different kernels on generative performance. We show that the quality of generated graphs is sensitive to the prior used, and that the optimal choice cannot be explained by obvious statistics or metrics, which challenges the intuitions which previous works have suggested.
- Structured denoising diffusion models in discrete state-spaces. Advances in Neural Information Processing Systems, 22:17981–17993, 7 2021. ISSN 10495258. doi: 10.48550/arxiv.2107.03006. URL https://arxiv.org/abs/2107.03006v2.
- Molgan: An implicit generative model for small molecular graphs. 5 2018. doi: 10.48550/arxiv.1805.11973. URL https://arxiv.org/abs/1805.11973v2.
- Diffusion models beat gans on image synthesis. Advances in Neural Information Processing Systems, 11:8780–8794, 5 2021. ISSN 10495258. doi: 10.48550/arxiv.2105.05233. URL https://arxiv.org/abs/2105.05233v4.
- Score-based generative modeling with critically-damped langevin diffusion. 12 2021. doi: 10.48550/arxiv.2112.07068. URL https://arxiv.org/abs/2112.07068v4.
- Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 2020-December, 6 2020. ISSN 10495258. doi: 10.48550/arxiv.2006.11239. URL https://arxiv.org/abs/2006.11239v2.
- Elucidating the design space of diffusion-based generative models. 6 2022. doi: 10.48550/arxiv.2206.00364. URL https://arxiv.org/abs/2206.00364v1.
- Priorgrad: Improving conditional denoising diffusion models with data-dependent adaptive prior. 6 2021. URL https://arxiv.org/abs/2106.06406v2.
- Efficient graph generation with graph recurrent attention networks. Advances in Neural Information Processing Systems, 32, 10 2019. ISSN 10495258. doi: 10.48550/arxiv.1910.00760. URL https://arxiv.org/abs/1910.00760v3.
- Spectre : Spectral conditioning helps to overcome the expressivity limits of one-shot graph generators. 4 2022. doi: 10.48550/arxiv.2204.01613. URL https://arxiv.org/abs/2204.01613v1.
- High-resolution image synthesis with latent diffusion models. 12 2021. doi: 10.48550/arxiv.2112.10752. URL https://arxiv.org/abs/2112.10752v2.
- Deep unsupervised learning using nonequilibrium thermodynamics. 32nd International Conference on Machine Learning, ICML 2015, 3:2246–2255, 3 2015. doi: 10.48550/arxiv.1503.03585. URL https://arxiv.org/abs/1503.03585v8.
- Score-based generative modeling through stochastic differential equations. 2021.
- Graphguide: interpretable and controllable conditional graph generation with discrete bernoulli diffusion. 2 2023. URL https://arxiv.org/abs/2302.03790v1.
- Digress: Discrete denoising diffusion for graph generation. 9 2022. doi: 10.48550/arxiv.2209.14734. URL https://arxiv.org/abs/2209.14734v1.
- Graphrnn: Generating realistic graphs with deep auto-regressive models. 35th International Conference on Machine Learning, ICML 2018, 13:9072–9081, 2 2018. doi: 10.48550/arxiv.1802.08773. URL https://arxiv.org/abs/1802.08773v3.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.
