Multi-view Graph Condensation via Tensor Decomposition (2508.14330v1)

Abstract: Graph Neural Networks (GNNs) have demonstrated remarkable results in various real-world applications, including drug discovery, object detection, social media analysis, recommender systems, and text classification. In contrast to their vast potential, training them on large-scale graphs presents significant computational challenges due to the resources required for their storage and processing. Graph Condensation has emerged as a promising solution to reduce these demands by learning a synthetic compact graph that preserves the essential information of the original one while maintaining the GNN's predictive performance. Despite their efficacy, current graph condensation approaches frequently rely on a computationally intensive bi-level optimization. Moreover, they fail to maintain a mapping between synthetic and original nodes, limiting the interpretability of the model's decisions. In this sense, a wide range of decomposition techniques have been applied to learn linear or multi-linear functions from graph data, offering a more transparent and less resource-intensive alternative. However, their applicability to graph condensation remains unexplored. This paper addresses this gap and proposes a novel method called Multi-view Graph Condensation via Tensor Decomposition (GCTD) to investigate to what extent such techniques can synthesize an informative smaller graph and achieve comparable downstream task performance. Extensive experiments on six real-world datasets demonstrate that GCTD effectively reduces graph size while preserving GNN performance, achieving up to a 4.0\ improvement in accuracy on three out of six datasets and competitive performance on large graphs compared to existing approaches. Our code is available at https://anonymous.4open.science/r/gctd-345A.

• The paper presents GCTD, which uses tensor decomposition to efficiently condense multi-view graphs for GNN training with minimal performance loss.
• It constructs a 3D tensor from multiple graph views and applies Tucker decomposition to synthesize a compact synthetic graph that preserves key structural information.
• Experimental results demonstrate enhanced accuracy and reduced resource utilization, offering scalable and interpretable graph condensation for large datasets.

Multi-view Graph Condensation via Tensor Decomposition

Introduction

The paper "Multi-view Graph Condensation via Tensor Decomposition" introduces a novel approach aimed at reducing the computational overhead associated with training Graph Neural Networks (GNNs) on large-scale graphs. This is achieved by transforming the graph condensation problem into a tensor decomposition problem, leveraging the capability of multi-view tensors to encapsulate graph information in a compressed form. The proposed method, referred to as GCTD, utilizes tensor decomposition to synthesize a much smaller graph that retains the essential predictive performance of the original graph.

Methodology

The primary innovation of GCTD lies in its use of tensor decomposition to model multi-view graphs. Traditional graph condensation methods often involve high computational costs due to multi-level optimization processes. However, GCTD circumvents these costs by utilizing a tensor representation, which simplifies the synthesis of smaller graphs.

Figure 1: Pipeline of GCTD. We construct a tensor by augmenting the graph's adjacency matrix $\mathbf{A}$.

1. Multi-view Augmentation: The process begins by creating multiple views of the graph through random edge perturbations. Each view provides a slightly altered perspective of the graph's structure, allowing the model to learn more robustly about the underlying data distributions.
2. Tensor Construction: These augmented views are stacked to form a 3D tensor. This tensor serves as the primary data structure for the decomposition, enabling the model to jointly consider multiple perspectives of the graph.
3. Tensor Decomposition: Employing a variation of the Tucker decomposition, the tensor is decomposed into latent factors that capture the essential co-clustering of nodes. This step effectively condenses the various perspectives into a consistent compact representation.
4. Synthetic Graph Generation: The reduced form generated by the decomposition is used to create a synthetic graph. This graph serves as a substitute for the original graph in GNN training, maintaining performance while significantly reducing computational demands.

Experimental Analysis

The experimental results demonstrate that GCTD performs favorably compared to existing graph condensation techniques, showing improvements in accuracy across several datasets. This indicates that the use of tensor decomposition not only preserves key structural and feature-related information but can also enhance interpretability and efficiency.

Figure 2: Accuracy scores achieved by our proposed method on graphs with varying numbers of views.

Figure 3: Visualization of condensed graphs generated by GCTD. Each node represents a synthetic node, with its color indicating the corresponding class.

• Performance on Various Datasets: The method was evaluated across a range of datasets, showing robustness in both small-scale and large-scale scenarios. For instance, GCTD outperformed traditional methods with improvements exceeding 4% in some cases.
• Execution Time and Resource Utilization: One of the key benefits of GCTD is its efficiency. The method significantly reduces the training time and memory requirements compared to traditional GNN training on full-scale graphs.

Implications and Future Work

The promising results of GCTD suggest several implications for future research and applications:

• Scalability: The reduction in resource requirement implies that GNNs can be applied to much larger datasets than was feasible previously, enabling their application in domains with hefty data needs such as bioinformatics and social network analysis.
• Interpretability: By linking the synthetic nodes directly back to their original counterparts through co-clustering, GCTD allows for more interpretable models, which is crucial for applications requiring explainability.
• Potential Enhancements: Future work could explore different tensor decomposition techniques or integrate additional data modalities to further improve the quality and applicability of synthetic graphs generated by GCTD.

Conclusion

GCTD represents a significant step forward in efficiently managing large-scale graph datasets for GNN applications. By leveraging tensor decomposition, it maintains or enhances graph neural network performance while mitigating computational constraints. This work sets the stage for more efficient and interpretable applications of graph-based machine learning models in various real-world scenarios.