BiGG Model: Scalable Graph Generation

• BiGG Model is a fully-autoregressive deep generative framework for constructing large, sparse graphs via recursive binary tree factorization.
• It leverages specialized TreeLSTM and Fenwick-tree structures to enable efficient parallel training and subquadratic scaling.
• The BiGG-E extension jointly models edge topology and real-valued weights, achieving state-of-the-art performance in diverse domains.

The BiGG (BIg Graph Generation) model is a fully-autoregressive deep generative framework for large-scale, sparse graph construction. Utilizing a recursive binary tree factorization over adjacency matrix rows and exploiting graph sparsity, BiGG achieves subquadratic generation time and memory scaling while retaining theoretical and empirical superiority over prior methods in both accuracy and scalability. The model's architecture enables fast, parallelizable training via specialized TreeLSTM and Fenwick-tree structures, and a principled extension to weighted graphs has resulted in BiGG-E, which jointly models edge topology and real-valued edge weights. These innovations have positioned BiGG and its derivatives as state-of-the-art for scalable, high-quality graph generative modeling on a wide spectrum of domains, including chemistry, social networks, and SAT instances (Dai et al., 2020, Williams et al., 30 Jul 2025).

1. Autoregressive Graph Generation and Model Factorization

BiGG targets the generative modeling of unattributed graphs $G = (V, E)$ with $n = |V|$ nodes and $m = |E|$ edges. A canonical node ordering $\pi$ is adopted, admitting a factorization

$p(G) \approx p(|V| = n) \cdot p(A^{\pi})$

where $A \in \{0,1\}^{n \times n}$ is the adjacency matrix. The fundamental principle is to avoid $\mathcal{O}(n^2)$ complexity by directly generating the $m$ present edges, not the dense $n \times n$ matrix. The core autoregressive decomposition is organized row-wise:

$$p(A) = \prod_{u=1}^n p(\mathcal{N}_u \mid \{\mathcal{N}_{u'}: u' < u\})$$

where $n = |V|$0 is the neighbor set for node $n = |V|$1. This structure allows BiGG to scale with the actual sparsity of the target graph (Dai et al., 2020, Williams et al., 30 Jul 2025).

2. Recursive Binary-Tree Decomposition and Algorithmic Structure

For each adjacency matrix row, BiGG constructs a full binary tree $n = |V|$2 over candidate targets $n = |V|$3, embedding each neighbor selection as a path in the tree. Internal nodes represent index intervals $n = |V|$4; two Bernoulli decisions at every node determine the existence of left ($n = |V|$5ch) and right (rch) children. The full tree likelihood is

$n = |V|$6

Recursive in-order tree traversal efficiently generates edges without repeating redundant adjacency or evaluation, ensuring scalability (Dai et al., 2020).

3. Neural Components and Architectural Innovations

Context propagation in BiGG is handled by TreeLSTM and LSTM modules:

• The top-down context $n = |V|$7 aggregates ancestor information.
• Each right child prediction conditions on $n = |V|$8 and a bottom-up TreeLSTM summary $n = |V|$9.
• After $m = |E|$0 is constructed, its root’s bottom-up state $m = |E|$1 forms a fixed-length embedding for row $m = |E|$2.
• These embeddings $m = |E|$3 are aggregated into a Fenwick-tree (binary indexed forest), producing the row history summary $m = |E|$4 in $m = |E|$5.

This design eliminates explicit instantiation of the $m = |E|$6 matrix, maintains only active intervals and necessary hidden state, and leverages paired TreeLSTM cells for efficient and rich contextualization. All probability outputs are Bernoulli distributions parameterized over the relevant state vectors (Dai et al., 2020).

4. Training Objective, Parallelization, and Computational Complexity

The maximum-likelihood training objective for BiGG is

$m = |E|$7

with $m = |E|$8 denoting ground-truth child presence.

Training is efficiently parallelized across the four computation stages—bottom-up TreeLSTM construction, Fenwick-forest row-wise aggregation, row-level LSTMs, and top-down likelihood computation—each requiring $m = |E|$9 synchronization. The aggregate generation cost is $\pi$0, reverting to $\pi$1 in the dense case, and memory usage is sublinear for optimally chosen block size $\pi$2, yielding $\pi$3 growth (Dai et al., 2020).

5. Extension to Weighted Graphs: The BiGG-E Model

BiGG-E generalizes BiGG to jointly generate weighted graphs $\pi$4 where $\pi$5 specifies edge weights, with $\pi$6 when $\pi$7. The factorization is

$\pi$8

where $\pi$9 and $p(G) \approx p(|V| = n) \cdot p(A^{\pi})$0, while $p(G) \approx p(|V| = n) \cdot p(A^{\pi})$1.

BiGG-E inherits all topological generation mechanisms of BiGG but maintains an additional Fenwick tree to summarize edge-weight embeddings. At each leaf of $p(G) \approx p(|V| = n) \cdot p(A^{\pi})$2, if $p(G) \approx p(|V| = n) \cdot p(A^{\pi})$3, the model predicts the parameters of the softplus-normal weight distribution via a small MLP applied to the merged topological and weight context. The weight's embedding is then inserted into the Fenwick tree for conditioning subsequent generations (Williams et al., 30 Jul 2025).

6. Empirical Evaluation and Applications

BiGG and BiGG-E have demonstrated state-of-the-art performance across several tasks:

• Benchmarks include protein graphs, 3D point clouds, large random grids, SAT instance literal-clause graphs, and classical random graphs.
• For topological metrics (MMD of degree/clustering/orbit counts, spectral distances), BiGG has matched or outperformed prior models by margins up to an order of magnitude.
• BiGG-E achieves best-in-class results for joint topology and weight metrics (e.g., MMD of weighted-degree or weighted Laplacian statistics), excelling especially for graph classes where edge weights exhibit strong mutual dependencies.
• Scalability experiments confirmed graph generation with $p(G) \approx p(|V| = n) \cdot p(A^{\pi})$4 nodes and $p(G) \approx p(|V| = n) \cdot p(A^{\pi})$5 edges, with both runtime and memory closely adhering to theoretical predictions. Competing models (e.g. Adj-LSTM, GRAN) are unable to scale beyond $p(G) \approx p(|V| = n) \cdot p(A^{\pi})$6 (Dai et al., 2020, Williams et al., 30 Jul 2025).

7. Limitations and Prospective Enhancements

Limitations identified for BiGG and BiGG-E include the growth of auxiliary tree structures (TreeLSTM, Fenwick) with graph size, and weight modeling flexibility constrained by the softplus-normal assumption. Other architectures may offer improved memory compression or better fit for heavy-tailed or structured weight distributions (e.g., normalizing flows). Extensions to categorical or vector-valued edge features, more elaborate conditioning (e.g., node attributes or global graph covariates), and integration with hybrid latent variable models are plausible directions for future work (Williams et al., 30 Jul 2025).

---

References

• (Dai et al., 2020) Scalable Deep Generative Modeling for Sparse Graphs
• (Williams et al., 30 Jul 2025) Scalable Generative Modeling of Weighted Graphs