Efficient & Degree-Guided Graph Generation (EDGE)

• The paper introduces a discrete diffusion-based framework that efficiently generates large, undirected graphs while preserving critical statistical properties.
• The methodology leverages active-node denoising and explicit degree guidance to control degree sequences and reduce computational complexity.
• Improvements in EDGE++ include degree-specific noise scheduling and volume-preserved sampling, which enhance graph fidelity and reduce GPU memory usage.

The Efficient and Degree-Guided Graph Generative Model (EDGE) is a discrete, diffusion-based framework for scalable, high-fidelity generation of large undirected graphs. By leveraging an absorbing discrete diffusion to the empty graph, explicit degree guidance, and active-node denoising, EDGE achieves state-of-the-art performance in both efficiency and the preservation of critical graph statistics. Subsequent developments, notably EDGE++, introduce degree-specific noise scheduling and volume-preserved sampling to further enhance computational scalability and graph fidelity (Chen et al., 2023, Wu et al., 2023).

1. Discrete Diffusion Formulation

EDGE operates on undirected simple graphs, representing each with its adjacency matrix $\bA \in \{0,1\}^{N\times N}$ (zero diagonal). The forward process defines a Markov kernel $q(\bA^{1:T}|\bA^0) = \prod_{t=1}^T q(\bA^{t}|\bA^{t-1})$, where at each step, edges are removed independently:

$q(\bA^t\,|\,\bA^{t-1}) = \prod_{i<j} \mathrm{Bern}\left(\bA^t_{ij};\, (1-\beta_t)\bA^{t-1}_{ij}\right)$

Here, $\beta_t$ is the removal probability, and $\bar\alpha_t = \prod_{\tau=1}^t (1-\beta_\tau)$. As $t$ increases, $\bA^T$ converges to the empty graph. The reverse process introduces two learnable distributions:

$p_\theta(\bA^{0:T},\bs^{1:T}) = p(\bA^T)\prod_{t=1}^T p_\theta(\bs^t\,|\,\bA^t)\, p_\theta(\bA^{t-1}|\bA^t, \bs^t)$

Here, $\bs^t \in \{0,1\}^N$ indicates the “active” nodes whose degree changed from $t-1 \to t$. At each reverse step, $\bs^t$ is sampled, and edges are added among active nodes according to parameterized Bernoulli distributions:

$p_\theta(\bA^{t-1}_{ij}|\bA^t, \bs^t) = \begin{cases} \mathrm{Bern}(\ell_{ij}^{t-1}) & \text{if } \bs^t_i = \bs^t_j = 1 \ 1_{\{\bA^{t-1}_{ij} = \bA^t_{ij}\}} & \text{otherwise} \end{cases}$

The objective is variation lower bound (VLB) maximization on $\log p_\theta(\bA^0)$, aligning each prediction step with the analytical backward kernel.

2. Sparsity and Computational Complexity

The diffusion is strictly edge-removing, never adding edges during the forward process. Thus, the expected edge count at step $t$ is:

$\mathbb{E}[M^t] = \bar\alpha_t M^0$

where $M^0$ is the initial edge count. In the reverse (generation) direction, only the subgraph induced by the current set of active nodes requires edge predictions at each time step, resulting in per-step work $O(M^t + (K^t)^2)$ with $K^t=\sum_i \bs^t_i \ll N$ typically. Over $T$ steps, the total complexity is

$O\left(T\max(M^0,\,\max_t (K^t)^2)\right)$

This contrasts sharply with prior diffusion-based models, which require $O(TN^2)$ per graph, making EDGE significantly more scalable for large sparse graphs (Chen et al., 2023).

3. Degree-Guided Generation

To match global statistics and control degree sequences in generated graphs, EDGE explicitly models the target degree vector $\bd^0 = \deg(\bA^0)$:

$p_\theta(\bA^0,\,\bd^0) = p_\theta(\bd^0) \, p_\theta(\bA^0 | \bd^0)$

Here, $p_\theta(\bd^0)$ is learned via RNN on empirical degree histograms, while the reverse process is conditioned on $\bd^0$. The active-node selection probability at each step is set to the analytic posterior:

$q(\bs^t_i=1 | d^t_i, d^0_i) = 1 - (1-\gamma_t)^{d^0_i - d^t_i}$

with $$\gamma_t = \frac{\beta_t \bar\alpha_{t-1}}{1-\bar\alpha_t}$$. This choice eliminates the KL divergence for node update and guarantees that sampled graphs match the desired degree statistics.

4. Neural Network Parameterization and Training

Both $p_\theta(\bs^t|\bA^t,\bd^0)$ and $p_\theta(\bA^{t-1}|\bA^t, \bs^t,\bd^0)$ share a GNN backbone. Node features include $(d^t_i, d^0_i)$ embedded via a multilayer perceptron (MLP), with a sinusoidal timestep embedding. The architecture stacks message-passing blocks (MPL + GRU), followed by global context MLPs, a node tag prediction head (for $\bs^t$), and a link prediction head (for edges):

$\ell_{ij}^{t-1} = \mathrm{MLP}(Z^L_i + Z^L_j)$

The loss is a sum of degree modeling error and per-step cross-entropy denoising error. Optimization uses Adam with learning rate $10^{-4}$, linear noise schedules, and practical batch sizes scaling with graph size.

5. Empirical Benchmarking and Results

EDGE has been evaluated on datasets of both moderate and large size, including Community, Ego, Polblogs, Cora, Road-MN, PPI, and the molecular QM9 corpus. Key evaluation metrics include:

• Small graphs: Maximum mean discrepancy (MMD) for degree, clustering, and orbits; FID and RBF MMD on graph embeddings.
• Large graphs: Power-law exponent (PLE), normalized triangle counts (NTC), clustering coefficient (CC), characteristic path length (CPL), assortativity coefficient (AC), and edge-overlap (EO).
• QM9: Validity, Uniqueness, Fréchet ChemNet Distance, and scaffold similarity.

Compared to GraphRNN, GRAN, GraphCNF, GDSS, DiscDDPM, DiGress, and edge-independent methods (OPB, HDOP, CELL, CO, TSVD, VGAE), EDGE achieved the lowest MMD on 8/10 metrics, superior neural scores, and generated statistics closest to real data for large graphs, with orders of magnitude speed-up over previous diffusion methods (Chen et al., 2023).

6. Edge++: Degree-Specific Noise Scheduling and Volume-Preserved Sampling

EDGE++ augments EDGE with two orthogonal improvements (Wu et al., 2023):

• Degree-specific noise scheduling: Rather than fixed noise schedules (e.g., linear over edges), EDGE++ directly controls the expected number of active nodes per step, smoothing compute/memory load. This is achieved via a binary search over the edge-removal probabilities $\alpha_{1:T}$, matched to a desired per-timestep active-node curve $g_{\gamma_t}(K) = K N \gamma_t$.
• Volume-preserved sampling: To address errors in satisfying the degree budget per step, EDGE++ introduces node and edge reweighting. The node reweighting enforces concentration of the sample total active node count to the desired expectation; edge reweighting sets the expected number of sampled edges to the analytic target, eliminating systematic drift and improving fidelity.

Empirical results show EDGE++ matches or betters EDGE on 7/8 structural statistics, while reducing GPU memory consumption during training by 31.3% (Polblogs) and 40.8% (PPI). Ablations confirm robustness to different $\gamma_t$ schedules, and explicit edge-overlap control becomes feasible.

7. Limitations, Ablations, and Future Directions

EDGE’s absorbing diffusion to $G(N,0)$ (empty graph) yields superior efficiency and MMD compared to reverse processes such as $G(N,0.5)$. The active-node strategy drastically reduces sampling time relative to full-node schemes, and explicit degree guidance closes MMD gaps. However, excessive diffusion steps $T$ for very sparse graphs can lead to few active nodes per step and potential class imbalance. Recommended $T$ is $64$–$256$. Future avenues include adopting more expressive GNN architectures, variance-reduction in training, and adaptive noise schedules (Chen et al., 2023, Wu et al., 2023).

Table: Summary of Major Improvements from EDGE to EDGE++

Enhancement	Effect on Statistics	Efficiency Impact
Degree-specific schedule	Matches desired node counts	31–41% less GPU memory
Volume-preserved sampling	No drift in degree/edge counts	Constant accuracy across EO

Both improvements are independent of the core network/training design and can be applied modularly to existing EDGE implementations.

• "Efficient and Degree-Guided Graph Generation via Discrete Diffusion Modeling" (Chen et al., 2023)
• "EDGE++: Improved Training and Sampling of EDGE" (Wu et al., 2023)