Score-Based Graph Generative Models

• Score-based Graph Generative Models are generative frameworks for graph data that use coupled SDEs to jointly model node features and graph structure.
• They address permutation invariance and node–edge dependencies, making them suitable for applications such as molecular generation and anomaly detection.
• Recent advances integrate permutation-equivariant networks, refined discretization techniques, and convergence analyses to overcome scalability and topology challenges.

Score-based Graph Generative Models (SGGMs) are score-based generative models for graph-structured data. A graph is represented as $G=(X,A)$, where $X\in\mathbb{R}^{N\times F}$ is the node feature matrix and $A\in\mathbb{R}^{N\times N}$ is the adjacency matrix. In contrast to common score-based generative models on Euclidean data, which are governed by a single stochastic differential equation (SDE), SGGMs involve a system of coupled SDEs on $(X,A)$, with separate yet interdependent dynamics for node features and graph structure. This coupled formulation underlies applications such as drug discovery, protein synthesis, molecular generation, and attributed-network anomaly detection, while also introducing graph-specific issues of permutation symmetry, node–edge dependence, and finite-step convergence (Su et al., 20 Aug 2025).

1. Representation, symmetry, and defining scope

Graphs are discrete, combinatorial objects whose underlying data distribution is invariant to the ordering of nodes. This makes permutation invariance a defining constraint for graph generative modeling. In one of the earliest score-based graph formulations, the score is modeled directly on adjacency matrices, and a permutation equivariant score network is used so that the induced graph distribution is permutation invariant; the central theorem is that a permutation equivariant score function implies a permutation invariant log-density (Niu et al., 2020).

This symmetry requirement is only one part of the graph-specific problem. Graphs with node attributes require modeling the dependency between nodes and edges rather than treating structure and features independently. The GDSS formulation addresses this by proposing a continuous-time framework in which the joint distribution of nodes and edges is modeled through a system of SDEs, with two partial scores, $$\nabla_{\mathbf{X}_t}\log p_t(\mathbf{X}_t,\mathbf{A}_t)$$ and $$\nabla_{\mathbf{A}_t}\log p_t(\mathbf{X}_t,\mathbf{A}_t)$$, learned by separate score networks (Jo et al., 2022).

Within this literature, two design lines are especially visible. The first models graph structure alone, typically on adjacency matrices with Gaussian perturbations and annealed Langevin dynamics. The second models the whole graph $G=(X,A)$ and makes the node–edge coupling explicit through partial scores and graph neural networks (GNNs). The second line is the basis of most contemporary SGGMs, including analyses of feature generation with fixed structure, structure generation with fixed features, and joint generation of both components (Su et al., 20 Aug 2025).

2. Coupled diffusion and reverse-time dynamics

A canonical continuous-time formulation treats the graph process $G_t=(X_t,A_t)$ through a coupled forward system

$$dX_t = f_X(G_t,t)\,dt + g_X(G_t,t)\,dW_t^X,\qquad dA_t = f_A(G_t,t)\,dt + g_A(G_t,t)\,dW_t^A,$$

with independent Brownian motions $W_t^X$ and $X\in\mathbb{R}^{N\times F}$0. A particularly simple and standard choice sets

$X\in\mathbb{R}^{N\times F}$1

so that both marginals are Ornstein–Uhlenbeck processes. Under this choice,

$X\in\mathbb{R}^{N\times F}$2

and each marginal converges exponentially to a standard Gaussian prior (Su et al., 20 Aug 2025).

The reverse-time process reconstructs data from the prior by following partial scores: $X\in\mathbb{R}^{N\times F}$3 This makes the distinction from Euclidean SGMs explicit: the relevant score object is not a single gradient field on $X\in\mathbb{R}^{N\times F}$4, but a pair of partial scores on the graph state space, and each partial score depends on the joint distribution over $X\in\mathbb{R}^{N\times F}$5 rather than only on one component (Su et al., 20 Aug 2025).

In GDSS, the forward diffusion is also written with a separable linear drift, which yields a factorized transition kernel

$X\in\mathbb{R}^{N\times F}$6

Because the conditional transitions are Gaussian, the training targets $X\in\mathbb{R}^{N\times F}$7 and $X\in\mathbb{R}^{N\times F}$8 are available in closed form. This makes denoising score matching tractable even when the target data distribution over graphs is unknown (Jo et al., 2022).

A recurrent practical pattern is continuous relaxation followed by discrete decoding. Binary or categorical graph structure is diffused in a continuous space, sampled with a reverse SDE or with annealed Langevin dynamics, and then quantized back to a graph. In adjacency-only models this appears as Gaussian perturbation plus thresholding of the final matrix; in joint node–edge models it appears as continuous diffusion of node and edge tensors followed by graph-specific decoding (Niu et al., 2020).

3. Score parameterization, graph networks, and samplers

SGGMs parameterize scores with permutation-equivariant graph architectures. In the adjacency-only setting, EDP-GNN is an edgewise dense prediction GNN with a multi-channel adjacency representation, nodewise message passing, and edgewise MLP updates. Its purpose is to output adjacency-shaped scores while preserving equivariance to node permutations, thereby inducing a permutation invariant graph distribution (Niu et al., 2020).

In joint node–edge models, the score functions are typically

$X\in\mathbb{R}^{N\times F}$9

and the dependence on graph structure is expressed through a GNN-like functional form. In the non-asymptotic convergence analysis, the score is assumed to be $A\in\mathbb{R}^{N\times N}$0-Lipschitz and representable as $A\in\mathbb{R}^{N\times N}$1, which makes the structural coupling explicit: the score depends on the product $A\in\mathbb{R}^{N\times N}$2, so node features and structure are interdependent (Su et al., 20 Aug 2025).

GDSS instantiates this coupling with a node score network built from stacked GNN layers and an edge score network using Graph Multi-Head Attention (GMH). The paper also introduces S4, a solver for the system of SDEs based on symmetric splitting and Langevin correction. S4 uses one score evaluation per time step and is presented as more efficient than standard predictor–corrector realizations that require separate evaluations for correction and prediction (Jo et al., 2022).

At the discretization level, two samplers are central in the recent theory: Euler–Maruyama (EM) and the exponential integrator (EI). EI exploits the linear Ornstein–Uhlenbeck part analytically and has smaller discretization error than EM; this difference appears explicitly in the finite-step KL bounds, and the practical recommendation is that EI should be preferred when accuracy is critical, whereas EM is more basic and may need smaller steps (Su et al., 20 Aug 2025).

Sampling acceleration has also become a separate research problem. Adaptive Momentum Sampling introduces a stochastic corrector based on adaptive momentum, motivated by the analogy between Langevin dynamics and stochastic gradient methods. It is designed to preserve randomness while accelerating sampling, and it is reported to produce more faithful images and graphs in small sampling steps with 2 to 5 times speed up and competitive scores compared to baselines on image and graph generation tasks (Wen et al., 2024).

4. Generation paradigms and non-asymptotic convergence

The first explicit non-asymptotic convergence analysis for SGGMs studies three graph generation paradigms: feature generation with fixed graph structure, graph structure generation with fixed node features, and joint generation of both graph structure and node features. The risk measure is Kullback–Leibler divergence between the true data distribution at time $A\in\mathbb{R}^{N\times N}$3 and the distribution induced by the discrete reverse process, for either $A\in\mathbb{R}^{N\times N}$4, $A\in\mathbb{R}^{N\times N}$5, or both. Small KL implies that sampled graphs approximate the true data distribution, and via Pinsker’s inequality this also controls total variation distance and, under bounded support, $A\in\mathbb{R}^{N\times N}$6 Wasserstein distance (Su et al., 20 Aug 2025).

Across all three paradigms, the bounds decompose generative error into a forward diffusion term, a score approximation term, and discretization terms. In schematic form,

$A\in\mathbb{R}^{N\times N}$7

Under uniform steps $A\in\mathbb{R}^{N\times N}$8, this simplifies to $A\in\mathbb{R}^{N\times N}$9 and $(X,A)$0, so discretization error is roughly $(X,A)$1 (Su et al., 20 Aug 2025).

A distinctive feature of the graph setting is the non-isotropic dependence on graph size and feature dimension. Terms involving adjacency scale like $(X,A)$2, whereas terms involving features scale like $(X,A)$3. The paper states that the performance of SGGMs deteriorates more significantly as the graph size increases compared to an increase in the feature dimensionality. This identifies a graph-specific bottleneck: increasing the number of nodes worsens convergence much faster than increasing the feature dimension, which explains empirical success on small graphs with rich features, such as molecules and small protein segments (Su et al., 20 Aug 2025).

The same analysis isolates topological effects. For feature generation with fixed structure, the bounds depend on $(X,A)$4, a bound on $(X,A)$5. Because spectral graph theory links this quantity to maximum degree, the analysis implies worse convergence for graphs with heterogeneous degree distributions than for more regular-like graphs. The paper therefore states that SGGMs are better at learning and generating graphs with uniform degree distributions than graphs with heterogeneous degree distributions such as power-law graphs (Su et al., 20 Aug 2025).

Feature magnitude enters similarly through $(X,A)$6. In the structure-generation setting, large feature norms increase KL risk, and the paper states that applying normalization can improve the convergence bound of SGGMs. This gives a theoretical basis for the standard use of feature normalization, and it extends to adjacency normalization or regularization through the appearance of $(X,A)$7 and the score Lipschitz constant $(X,A)$8 in the bounds (Su et al., 20 Aug 2025).

The same theory also gives practical prescriptions for the diffusion length $(X,A)$9 and the number of sampling steps $$\nabla_{\mathbf{X}_t}\log p_t(\mathbf{X}_t,\mathbf{A}_t)$$0. In the joint-generation case with EI and uniform steps, choosing

$$\nabla_{\mathbf{X}_t}\log p_t(\mathbf{X}_t,\mathbf{A}_t)$$1

ensures that the KL risk is dominated by score error. The operative trade-off is that longer diffusion reduces forward OU error but increases both the $$\nabla_{\mathbf{X}_t}\log p_t(\mathbf{X}_t,\mathbf{A}_t)$$2 term and the number of steps required to keep discretization error small (Su et al., 20 Aug 2025).

5. Empirical domains, diagnostics, and limitations

In graph generation, GDSS is evaluated on generic graphs such as Ego-small, Community-small, Enzymes, and Grid, and on molecular datasets such as QM9 and ZINC250k. The reported pattern is that the method either achieves significantly superior or competitive performance to baselines, while also modeling node–edge relationships sufficiently well that generated molecules lie close to the training distribution yet do not violate the chemical valency rule. On QM9 and ZINC250k, GDSS achieves 95–97% validity w/o correction, which is presented as evidence that validity can emerge from learning the joint node–edge distribution rather than from explicit hard constraints (Jo et al., 2022).

A different application uses SGGMs as local generative priors for anomaly detection in attributed networks. In that setting, the network is decomposed into ego-graphs, GDSS is trained on the ego-graph distribution, and node anomaly scores are defined through reconstruction error across multiple diffusion noise levels. The reported outcome is competitive results on small-scale graphs, together with an explicit empirical analysis of Dirichlet energy showing that generative models might struggle to accurately reconstruct it (Gavrilev et al., 2023).

The Dirichlet-energy analysis exposes a specific limitation of graph diffusion models. Reconstructed ego-graphs tend to have lower normalized Dirichlet energy than the originals, and this bias becomes more pronounced as noise level increases. In practical terms, energy-based anomaly scoring is generally worse than matrix distance except on one dataset, and the result is interpreted as a tendency toward smoother or sparser reconstructions. This identifies a spectral failure mode that is not visible from entrywise adjacency or feature errors alone (Gavrilev et al., 2023).

Scalability remains a recurrent constraint. In attributed-network anomaly detection, adjacency score networks scale as $$\nabla_{\mathbf{X}_t}\log p_t(\mathbf{X}_t,\mathbf{A}_t)$$3 per ego-graph, so high-degree ego-graphs must be truncated to remain trainable. On the DGraph benchmark, the implementation hits time or memory limits, and the method cannot be processed at true large scale without further architectural and algorithmic changes (Gavrilev et al., 2023). This limitation is consistent with the finite-step convergence theory, where adjacency-related terms scale as $$\nabla_{\mathbf{X}_t}\log p_t(\mathbf{X}_t,\mathbf{A}_t)$$4 and dominate feature-related terms as graph size grows (Su et al., 20 Aug 2025).

Controlled synthetic studies reinforce the same picture. Experiments with regular graphs and Barabási–Albert power-law graphs show that KL divergence grows faster for power-law graphs than for regular graphs, that applying normalization to node features significantly reduces KL divergence, and that performance deteriorates more rapidly with increasing $$\nabla_{\mathbf{X}_t}\log p_t(\mathbf{X}_t,\mathbf{A}_t)$$5 than with increasing $$\nabla_{\mathbf{X}_t}\log p_t(\mathbf{X}_t,\mathbf{A}_t)$$6. These empirical findings align with the explicit graph-topological dependencies in the convergence bounds (Su et al., 20 Aug 2025).

6. Related theoretical perspectives and open directions

Several theoretical perspectives surrounding SGGMs are presently indirect rather than graph-native. A geometric analysis of score-based generative models shows that both the forward noising process and the reverse generative process can be viewed as Wasserstein gradient flows in the space of probability measures. For continuous graph representations or latent graph embeddings, this suggests a natural way to reinterpret SGGMs as gradient flows on distributions over graph encodings rather than only as reverse-time SDE simulators (Ghimire et al., 2023).

A second adjacent perspective models score approximation error as a stochastic source in a forward SPDE for the density field. That work does not discuss graphs explicitly, but it develops a general viewpoint in which score error acts as a stochastic divergence term and proposes a quadratic-variation-based evaluation metric. A plausible implication is that, for SGGMs operating on continuous graph encodings, score error could be analyzed at the level of evolving graph-density fields rather than only along sample trajectories (Seo, 9 Feb 2026).

Generalization theory remains much less developed for graphs than for Euclidean SGMs. Recent results establish minimax convergence rates for SGMs in $$\nabla_{\mathbf{X}_t}\log p_t(\mathbf{X}_t,\mathbf{A}_t)$$7-Wasserstein distance under denoising score matching, and separate analyses show dimension-independent rates for families of sub-Gaussian distributions whose log-relative density admits bounded-complexity neural approximation. These papers do not work with graphs directly, but they suggest that analogous SGGM theory may require continuous graph embeddings, regularity assumptions on graph scores, and function-approximation results for GNNs on graph representation spaces (Stéphanovitch et al., 7 Jul 2025); (Cole et al., 2024).

Wasserstein convergence guarantees for general SGMs with smooth log-concave data distributions also indicate that the choice of forward process affects iteration complexity, and that VP-like schedules can outperform some VE-like schedules. For SGGMs built on continuous graph representations, this suggests that forward-process design should be treated as a first-class modeling choice rather than a fixed implementation detail (Gao et al., 2023).

The graph-specific theoretical agenda is correspondingly clear. Open directions already identified in the non-asymptotic convergence analysis include incorporating the dynamics of the learning process rather than only static score error, refining analysis under specific generative graph models such as contextual stochastic block models, and adapting the tools to discrete-time graph diffusion such as Digress. More broadly, the field is moving toward an overview in which permutation symmetry, coupled node–edge diffusion, graph topology, finite-step sampling, and score-network regularity are analyzed within a common framework rather than as separate implementation concerns (Su et al., 20 Aug 2025).