DeFoG: Discrete Flow Matching for Graph Generation (2410.04263v3)

Abstract: Graph generative models are essential across diverse scientific domains by capturing complex distributions over relational data. Among them, graph diffusion models achieve superior performance but face inefficient sampling and limited flexibility due to the tight coupling between training and sampling stages. We introduce DeFoG, a novel graph generative framework that disentangles sampling from training, enabling a broader design space for more effective and efficient model optimization. DeFoG employs a discrete flow-matching formulation that respects the inherent symmetries of graphs. We theoretically ground this disentangled formulation by explicitly relating the training loss to the sampling algorithm and showing that DeFoG faithfully replicates the ground truth graph distribution. Building on these foundations, we thoroughly investigate DeFoG's design space and propose novel sampling methods that significantly enhance performance and reduce the required number of refinement steps. Extensive experiments demonstrate state-of-the-art performance across synthetic, molecular, and digital pathology datasets, covering both unconditional and conditional generation settings. It also outperforms most diffusion-based models with just 5-10% of their sampling steps.

• The paper introduces a novel discrete flow matching framework that employs graph transformers and RRWP to generate complex graph structures.
• It decouples training from sampling using a flow-based method, enhancing flexibility and reducing computational costs compared to traditional diffusion models.
• Empirical results on synthetic and molecular datasets, backed by theoretical guarantees, demonstrate state-of-the-art performance and fidelity to true data distributions.

Overview of DeFoG: Discrete Flow Matching for Graph Generation

The paper presents a novel framework, DeFoG, which leverages discrete flow matching (DFM) for graph generation, addressing limitations inherent in diffusion models related to sampling efficiency and flexibility. Graph generation is crucial across various scientific domains due to its capability to interpret complex data structures and create new, realistic samples. By employing a flow-based methodology, DeFoG incorporates a linear interpolation noising process alongside a continuous-time Markov chain (CTMC) denoising process. These innovations decouple training and sampling, facilitating optimized model performance.

Key Contributions

A significant contribution of DeFoG is its introduction of a DFM model tailored to graphs, utilizing graph transformers while ensuring node permutation properties to preserve graph symmetry. The model's framework allows for extensive algorithmic enhancements to boost performance, providing a strong theoretical basis through rigorous analysis confirming the model's ability to replicate the true data distribution.

Empirically, DeFoG demonstrates state-of-the-art outcomes, outperforming diffusion models on both synthetic and molecular datasets. The paper elaborates on DeFoG's efficacy in tasks involving conditional generation, particularly in digital pathology contexts.

Algorithmic Innovations

Key advancements in DeFoG include:

• Architectural Design: Utilization of a graph transformer to manage the graph-to-graph mapping which is fundamental for generating intricate node and edge structures, respecting intrinsic graph properties.
• Enhanced Expressivity: The model employs Relative Random Walk Probabilities (RRWP) to amplify its representational power, crucial for addressing limitations traditionally faced by graph neural networks.
• Training and Sampling Flexibility: The decoupled nature of DeFoG allows the disentanglement of training procedures from sampling processes, enabling the use of variable sampling step sizes and adaptive strategies which are crucial for dataset-specific optimization.

Theoretical Guarantees

The theoretical robustness of DeFoG is underscored by:

1. A bound on the estimation error of the multivariate rate matrix, related to model design choices.
2. A bounded deviation of generated distributions from the ground truth, ensuring the reliability of the independent-dimensional Euler method for CTMC simulation.

These guarantees confirm that the model maintains fidelity to the underlying data distribution, validating architectural and procedural design decisions.

Implications and Future Directions

DeFoG, by bridging gaps left by traditional models, enhances the efficiency and effectiveness of graph generation tasks, opening opportunities for deploying generative models across various domains. Practically, the model’s ability to manage complex datasets with reduced computational costs highlights its potential for broader applications in fields such as molecular biology and digital pathology.

Looking forward, DeFoG's flexible framework may inspire further exploration into adaptive strategies for diverse graph structures, potentially leading to developments in graph-related AI applications, particularly those involving dynamic, high-dimensional networks. Additionally, exploring integration into real-time data streams could present a valuable advancement, aiding in rapid decision-making processes in data-intensive fields.