Efficient Graph Generation with Graph Recurrent Attention Networks (1910.00760v3)

Abstract: We propose a new family of efficient and expressive deep generative models of graphs, called Graph Recurrent Attention Networks (GRANs). Our model generates graphs one block of nodes and associated edges at a time. The block size and sampling stride allow us to trade off sample quality for efficiency. Compared to previous RNN-based graph generative models, our framework better captures the auto-regressive conditioning between the already-generated and to-be-generated parts of the graph using Graph Neural Networks (GNNs) with attention. This not only reduces the dependency on node ordering but also bypasses the long-term bottleneck caused by the sequential nature of RNNs. Moreover, we parameterize the output distribution per block using a mixture of Bernoulli, which captures the correlations among generated edges within the block. Finally, we propose to handle node orderings in generation by marginalizing over a family of canonical orderings. On standard benchmarks, we achieve state-of-the-art time efficiency and sample quality compared to previous models. Additionally, we show our model is capable of generating large graphs of up to 5K nodes with good quality. To the best of our knowledge, GRAN is the first deep graph generative model that can scale to this size. Our code is released at: https://github.com/lrjconan/GRAN.

Efficient Graph Generation with Graph Recurrent Attention Networks

This paper presents a novel approach to efficient and expressive graph generation, introducing Graph Recurrent Attention Networks (GRANs). The proposed model addresses several challenges associated with existing methods by leveraging advanced techniques in Graph Neural Networks (GNNs) and attention mechanisms, making significant strides towards improved efficiency and quality in graph generation.

Overview of GRAN

GRAN operates by generating graphs one block of nodes and associated edges at a time, allowing users to adjust the block size and sampling stride to balance sample quality against efficiency. This method contrasts with previous approaches that rely heavily on Recurrent Neural Networks (RNNs), which often suffer from issues related to sequential node ordering and scalability.

Key features of GRAN include:

1. Auto-Regressive Conditioning: Using GNNs with attention mechanisms, GRAN achieves superior modeling of dependencies between generated graph segments. This reduces dependency on node ordering and mitigates the performance bottleneck caused by the sequential nature of RNNs.
2. Mixture of Bernoulli Distributions: The output distribution per block is parameterized using a mixture of Bernoulli distributions. This approach captures correlations among generated edges within a block, providing a more flexible and expressive model.
3. Node Ordering: GRAN proposes handling node orderings by marginalizing over a family of canonical orderings. This helps accommodate different ordering biases, optimizing the likelihood computation across possible permutations.

Performance and Implications

The paper reports state-of-the-art performance on benchmark datasets, demonstrating notable improvements in both time efficiency and sample quality compared to existing models. The ability to generate large graphs with up to 5K nodes with good quality further underscores GRAN's scalability and robustness.

Numerical Results

GRAN achieves impressive results across several datasets, as highlighted by better maximum mean discrepancy (MMD) scores for degree distributions, clustering coefficients, and other graph statistics. Using a mixture of Bernoulli distributions allows the model to efficiently capture complex dependencies, enhancing the overall quality of the generated graphs.

Implications for Future Research

The introduction of GRAN opens up several avenues for future exploration in AI and graph-related domains:

• Scalable Graph Generation: The combination of GNNs and attention mechanisms offers a promising direction for scaling graph generation to even larger datasets in diverse application areas like social networks and biological systems.
• Applications in Molecular and Network Sciences: With its capacity to model complex dependencies and generate high-quality graphs, GRAN could significantly impact fields such as drug design and network architecture search.
• Exploration of Canonical Orderings: Further research could investigate optimal combinations of canonical orderings tailored for specific domains, potentially enhancing model performance and efficiency.

Conclusion

GRAN represents a sophisticated advancement in the field of graph generative models, addressing key limitations of older methods by utilizing state-of-the-art neural network techniques. Its flexibility in balancing efficiency and quality, paired with its capability to handle large graphs, makes it a valuable tool for researchers and practitioners working with complex graph data.