Graph Neural Networks with Diverse Spectral Filtering (2312.09041v3)

Abstract: Spectral Graph Neural Networks (GNNs) have achieved tremendous success in graph machine learning, with polynomial filters applied for graph convolutions, where all nodes share the identical filter weights to mine their local contexts. Despite the success, existing spectral GNNs usually fail to deal with complex networks (e.g., WWW) due to such homogeneous spectral filtering setting that ignores the regional heterogeneity as typically seen in real-world networks. To tackle this issue, we propose a novel diverse spectral filtering (DSF) framework, which automatically learns node-specific filter weights to exploit the varying local structure properly. Particularly, the diverse filter weights consist of two components -- A global one shared among all nodes, and a local one that varies along network edges to reflect node difference arising from distinct graph parts -- to balance between local and global information. As such, not only can the global graph characteristics be captured, but also the diverse local patterns can be mined with awareness of different node positions. Interestingly, we formulate a novel optimization problem to assist in learning diverse filters, which also enables us to enhance any spectral GNNs with our DSF framework. We showcase the proposed framework on three state-of-the-arts including GPR-GNN, BernNet, and JacobiConv. Extensive experiments over 10 benchmark datasets demonstrate that our framework can consistently boost model performance by up to 4.92% in node classification tasks, producing diverse filters with enhanced interpretability. Code is available at \url{https://github.com/jingweio/DSF}.

• The paper introduces a Diverse Spectral Filtering framework that learns node-specific filter weights to capture both global and local graph features.
• It employs an optimization method by embedding graph vertices into a low-dimensional space to adjust homogeneous spectral filters to diverse local contexts.
• Experiments show the DSF framework significantly improves node classification accuracy and offers deeper interpretability on complex, non-uniform graphs.

Understanding Graph Neural Networks with Diverse Spectral Filtering

Introduction to Spectral GNNs

Graph Neural Networks (GNNs) have become a critical tool for learning from graph-structured data across various fields, from social networks to recommendation systems. Among them, spectral GNNs play a pivotal role. They operate by transforming input features into a new space through a process akin to convolution operations in the spectral domain. Despite their success, traditional spectral GNNs encounter difficulties when dealing with complex networks due to their one-size-fits-all approach in spectral filtering, which overlooks the uniqueness of the local structures in the network.

Diverse Spectral Filtering Framework

To address these limitations, a novel framework, Diverse Spectral Filtering (DSF), is introduced. Its core idea is to learn node-specific filter weights that can accommodate the diversity of local structures across different regions of the graph. The DSF framework achieves this by combining a globally shared filter with locally varying filter weights that reflect node differences driven by distinct parts of the graph. This balancing act enables the DSF framework to capture both global characteristics of the graph and the nuanced variations across different nodes, leading to enhanced interpretability and consistently improved performance in node classification tasks.

Optimizing Spectral GNNs

To optimize the learning of diverse filters, a new optimization problem is designed. It starts with embedding graph vertices into a low-dimensional space that encodes the positional information of nodes. Building upon this, node-specific coefficients are determined in a way that adjusts the original homogeneous spectral filters to the node’s local context. This refinement process creates a graph filter that is sensitive to individual nodes' unique positions, facilitating a more nuanced understanding of their diverse local contexts.

Enhancement of Existing Models and Interpretability

By plugging the DSF framework into existing spectral GNNs, it's been shown to boost model performance significantly, notably on datasets with non-uniform graphs. Experiments conducted across multiple benchmark datasets confirm that the DSF framework not only enhances model accuracy but also provides improved interpretability. It's capable of differentiating between the global structure and the local peculiarities of nodes positioned in diverse graph regions.

Conclusion

The DSF framework represents a significant advancement in spectral GNNs. By breaking free from the constraints of homogeneous filtering and embracing the diversity of local graph structures, DSF sets a new standard in graph machine learning, offering superior performance and richer interpretative insights. This groundbreaking framework is available for the community to experiment with and integrate into their own spectral GNN models, promising a new horizon for research and applications in graph-based learning systems.