Fully-inductive Node Classification on Arbitrary Graphs (2405.20445v5)

Abstract: One fundamental challenge in graph machine learning is generalizing to new graphs. Many existing methods following the inductive setup can generalize to test graphs with new structures, but assuming the feature and label spaces remain the same as the training ones. This paper introduces a fully-inductive setup, where models should perform inference on arbitrary test graphs with new structures, feature and label spaces. We propose GraphAny as the first attempt at this challenging setup. GraphAny models inference on a new graph as an analytical solution to a LinearGNN, which can be naturally applied to graphs with any feature and label spaces. To further build a stronger model with learning capacity, we fuse multiple LinearGNN predictions with learned inductive attention scores. Specifically, the attention module is carefully parameterized as a function of the entropy-normalized distance features between pairs of LinearGNN predictions to ensure generalization to new graphs. Empirically, GraphAny trained on a single Wisconsin dataset with only 120 labeled nodes can generalize to 30 new graphs with an average accuracy of 67.26%, surpassing not only all inductive baselines, but also strong transductive methods trained separately on each of the 30 test graphs.

• The paper introduces GraphAny, a foundation model that integrates LinearGNN with an inductive attention module to enable inductive node classification on any graph.
• It employs a closed-form solution in LinearGNN, eliminating the need for extensive training while normalizing outputs through entropy-based attention.
• Empirical results on 31 datasets demonstrate GraphAny’s superior generalization with 67.26% average accuracy and nearly 3x faster runtime than conventional GNNs.

An Analysis of GraphAny: A Foundation Model for Node Classification on Any Graph

The paper "GraphAny: A Foundation Model for Node Classification on Any Graph" addresses a fundamental challenge in the domain of graph-based machine learning. Traditional methods typically require models to be trained on graphs with specific feature and label spaces, limiting their utility for inductive inference tasks where graphs with different feature and label spaces are encountered. This paper introduces a novel framework, GraphAny, engineered to overcome these limitations and perform reliable node classification on any graph using a foundation model approach.

GraphAny Architecture

GraphAny comprises two primary components: LinearGNN and an inductive attention module. The LinearGNN performs inference on new graphs by deriving an analytical solution, thereby circumventing the need for extensive training on each new graph. The inductive attention module synthesizes predictions from multiple LinearGNNs, using carefully parameterized entropy-normalized distance features, to ensure generalized performance across varying graphs.

1. LinearGNN:

The LinearGNN component is based on simple yet effective graph convolution operations. By modeling the mapping between node features and labels as a non-parametric graph convolution followed by a linear layer, LinearGNN can provide a closed-form solution for optimal weight determination. This design enables the LinearGNN to process new graphs efficiently as it eliminates the need for gradient-based optimization typically required in conventional GNN training.

2. Inductive Attention Module:

The attention module in GraphAny utilizes entropy-normalized distance features to combine the outputs of multiple LinearGNNs. This is crucial for:

• Ensuring that the attention mechanism is invariant to feature and label permutations.
• Normalizing feature distributions to account for variations in label dimensions across different graphs. These characteristics empower GraphAny to achieve strong generalization performance, as the attention module can dynamically adapt to the structure and attributes of new graphs.

Empirical Validation and Performance

The empirical evaluation of GraphAny is conducted on a diverse set of 31 node classification datasets. These datasets range from small academic networks to large-scale e-commerce and social networks, ensuring a comprehensive assessment of the model's robustness. Four GraphAny models, each trained on a different dataset (Cora, Wisconsin, Arxiv, Products), demonstrate remarkable inductive inference capabilities, often surpassing traditional GCN and GAT models trained in a supervised manner for each dataset individually.

Key findings include:

• Inductive Generalization: GraphAny significantly outperforms other inductive baselines and non-parametric methods in terms of accuracy. Specifically, GraphAny trained on the Wisconsin dataset achieved an average accuracy of 67.26% across 30 new graphs, highlighting its ability to generalize beyond the training graph.
• Efficiency: The time complexity analysis reveals that GraphAny is more efficient than conventional GNNs, owing to the elimination of training steps and the use of preprocessed graph convolution results. This results in a nearly 3x reduction in total runtime compared to a GCN.

Insights from Visualization

Attention weight visualization offers insights into how GraphAny combines various LinearGNN models. The attention mechanism effectively identifies and prioritizes the most suitable LinearGNN for each graph, demonstrating adaptability. Interestingly, attention patterns vary depending on the dataset used for training, reflecting the inherent properties of the training data. This adaptability is a key strength of GraphAny, allowing it to maintain high performance across diverse graph structures and types.

Future Directions and Implications

Theoretical and Practical Impacts:

GraphAny sets a precedent for the development of foundation models tailored to graph-structured data. It opens new avenues for research into more expressive LinearGNN variants and sophisticated attention mechanisms that can handle even more complex graph tasks, including edge-level and graph-level predictions.

Future Work:

Future research may explore expanding GraphAny to handle regression tasks and relational graphs. Additionally, improving the expressiveness of LinearGNNs and adapting GraphAny to integrate with various graph neural architectures could further enhance its generalization and efficiency.

In conclusion, GraphAny represents a significant advancement in the field of graph-based machine learning, providing a scalable and effective solution for inductive node classification. It paves the way for robust and versatile graph foundation models that can be applied to real-world datasets with minimal retraining, thereby offering substantial utility in diverse applications across industry and academia.