Edge-Splitting MLP: Node Classification on Homophilic and Heterophilic Graphs without Message Passing (2412.08310v1)

Abstract: Message Passing Neural Networks (MPNNs) have demonstrated remarkable success in node classification on homophilic graphs. It has been shown that they do not solely rely on homophily but on neighborhood distributions of nodes, i.e., consistency of the neighborhood label distribution within the same class. MLP-based models do not use message passing, \eg Graph-MLP incorporates the neighborhood in a separate loss function. These models are faster and more robust to edge noise. Graph-MLP maps adjacent nodes closer in the embedding space but is unaware of the neighborhood pattern of the labels, i.e., relies solely on homophily. Edge Splitting GNN (ES-GNN) is a model specialized for heterophilic graphs and splits the edges into task-relevant and task-irrelevant, respectively. To mitigate the limitations of Graph-MLP on heterophilic graphs, we propose ES-MLP that combines Graph-MLP with an edge-splitting mechanism from ES-GNN. It incorporates the edge splitting into the loss of Graph-MLP to learn two separate adjacency matrices based on relevant and irrelevant feature pairs. Our experiments on seven datasets with six baselines show that ES-MLP is on par with homophilic and heterophilic models on all datasets without using edges during inference. We show that ES-MLP is robust to multiple types of edge noise during inference and that its inference time is two to five times faster than that of commonly used MPNNs. The source code is available at https://github.com/MatthiasKohn/ES-MLP.

• The paper presents a novel Edge-Splitting MLP that bypasses message passing by partitioning edges into task-relevant and task-irrelevant sets.
• It integrates a neighborhood contrastive loss to capture graph structure, achieving competitive results on both homophilic and heterophilic datasets.
• The approach significantly reduces inference time while demonstrating robustness to edge noise, advancing scalable graph learning.

Edge-Splitting MLP: Node Classification on Homophilic and Heterophilic Graphs without Message Passing

In the domain of graph-based machine learning, Graph Neural Networks (GNNs) are known for their proficiency in tasks such as node classification, with the foundational technique being Message Passing Neural Networks (MPNNs). However, these models often assume a homophilic graph structure, which is not universally applicable since many real-world graphs are heterophilic in nature, where connected nodes have differing labels. The research paper "Edge-Splitting MLP: Node Classification on Homophilic and Heterophilic Graphs without Message Passing" addresses this limitation by proposing a model called Edge-Splitting MLP (ES-MLP), which integrates properties of Graph-MLP and ES-GNN. This model innovatively bypasses the message-passing mechanism prevalent in GNNs while accommodating both homophilic and heterophilic properties of graphs.

Core Concepts and Methodology

The Edge-Splitting MLP (ES-MLP) is designed to operate efficiently on both homophilic and heterophilic graphs without incurring the computational overheads associated with message passing. It leverages the edge-splitting concept from ES-GNN, which partitions graph edges into task-relevant and task-irrelevant sets, an essential step in handling heterophilic structures. Additionally, ES-MLP employs a neighborhood contrastive loss from Graph-MLP, further refining node classification without relying on the traditional adjacency matrix during inference.

Key features of the ES-MLP model include:

• Edge Splitting: The model differentiates edges into relevant and irrelevant categories, allowing for selective aggregation that enhances task performance on heterophilic graphs.
• Neighborhood Contrastive Loss: This loss function, adapted from Graph-MLP, ensures that the learning process incorporates graph structure information effectively, particularly in task-relevant subspaces.
• Increased Computational Efficiency: Without the necessity for message passing, ES-MLP facilitates faster inference times, evidenced by performance evaluations showing it to be two to five times faster than standard MPNNs.

Experimental Results and Observations

ES-MLP's efficacy was rigorously tested on seven real-world datasets and one synthetic dataset generated using the Contextual Stochastic Block Model (CSBM). The datasets varied in homophily, allowing for a comprehensive evaluation. Key findings from the experiments include:

• Performance: ES-MLP effectively competes with state-of-the-art models on both homophilic and heterophilic datasets. It outperforms baseline models on heterophilic datasets such as Amazon and Roman, demonstrating the model's strength in diverse conditions.
• Robustness to Edge Noise: The model demonstrated robust performance under different types of edge noise, crucial for real-world applications where graph data can be unreliable. Unlike MPNNs, which display performance degradation when faced with noise, ES-MLP's architecture remains stable.
• Inference Speed: The practical advantage of reduced inference time is notable, with ES-MLP requiring significantly less time compared to traditional GNNs, attributed to its avoidance of message passing.

Theoretical and Practical Implications

This research introduces a technically sophisticated approach for node classification on graphs with varying structural properties. The ES-MLP model's lack of reliance on message passing not only facilitates reduced computational demands but also offers resilience against data inconsistencies, marking a significant advancement in graph learning methodologies. The theoretical implications extend to improved model generalizability across differing homophily ratios, while practically, it offers more efficient deployments in applications like social network analysis and fraud detection in heterophilic connectivity scenarios.

Future Directions

The paper proposes extending ES-MLP by modeling graph directionality, which may further enhance its capabilities on heterophilic datasets. Additionally, exploring different architectures or adjoint frameworks that could leverage such edge-splitting methods might offer further optimization potential.

In conclusion, the ES-MLP represents a significant contribution to handling heterophilic data structures within the scope of graph-based learning. By decoupling from the message-passing paradigm and optimizing node embeddings through innovative losses, it suggests a new pathway in scalable and robust graph learning models.