Large-Scale Learnable Graph Convolutional Networks (1808.03965v1)

Abstract: Convolutional neural networks (CNNs) have achieved great success on grid-like data such as images, but face tremendous challenges in learning from more generic data such as graphs. In CNNs, the trainable local filters enable the automatic extraction of high-level features. The computation with filters requires a fixed number of ordered units in the receptive fields. However, the number of neighboring units is neither fixed nor are they ordered in generic graphs, thereby hindering the applications of convolutional operations. Here, we address these challenges by proposing the learnable graph convolutional layer (LGCL). LGCL automatically selects a fixed number of neighboring nodes for each feature based on value ranking in order to transform graph data into grid-like structures in 1-D format, thereby enabling the use of regular convolutional operations on generic graphs. To enable model training on large-scale graphs, we propose a sub-graph training method to reduce the excessive memory and computational resource requirements suffered by prior methods on graph convolutions. Our experimental results on node classification tasks in both transductive and inductive learning settings demonstrate that our methods can achieve consistently better performance on the Cora, Citeseer, Pubmed citation network, and protein-protein interaction network datasets. Our results also indicate that the proposed methods using sub-graph training strategy are more efficient as compared to prior approaches.

Overview of "Large-Scale Learnable Graph Convolutional Networks"

The paper introduces a novel approach to apply convolutional neural networks (CNNs) on graph data through the proposed Learnable Graph Convolutional Layer (LGCL). Traditional CNNs excel with grid-like data but face challenges with graph structures due to the variability in node neighborhoods and the absence of ordering. This work presents a method to overcome these difficulties by transforming graphs into a format more amenable to CNN methods.

Methodology

The key innovation within this work is the LGCL, which effectively adapts CNNs to graph data:

• Graph Transformation: The LGCL includes a novel $k$-largest node selection process, which transforms graphs into 1-D grid-like structures using ranking of node feature values. This transformation allows the subsequent application of conventional convolutional operations.
• Sub-graph Training Strategy: To address the substantial computational and memory demands of training on large graphs, a sub-graph training method is introduced. This strategy efficiently samples sub-graphs for each training iteration, reducing resource requirements while maintaining performance.

Experimental Evaluation

Experiments were conducted on several datasets, including Cora, Citeseer, Pubmed, and the PPI dataset to validate the effectiveness of the proposed methods:

• Transductive Learning Results: The LGCL-based models demonstrated superior performance compared to state-of-the-art graph convolutional networks (GCNs), with improvements of 1.8%, 2.7%, and 0.6% on the Cora, Citeseer, and Pubmed datasets respectively.
• Inductive Learning Results: On the PPI dataset, the proposed models achieved an F1 score improvement of 16% over existing methods, highlighting robust generalization capabilities even when test data structures are unseen during training.
• Training Efficiency: The sub-graph training strategy significantly accelerated the training process compared to whole-graph approaches, with negligible performance degradation. This method proves particularly beneficial for large-scale data where resource constraints are critical.

Implications and Future Work

The results demonstrate that transforming graphs into grid-like structures via LGCL allows for direct application of CNNs on graphs, thereby harnessing their feature extraction capabilities. The sub-graph method not only improves efficiency but also suggests potential scalability for large and complex graph applications.

Potential future developments could explore:

• Graph Classification: Extending the methodologies to graph classification tasks and developing mechanisms for down-sampling graphs akin to pooling operations in image processing.
• Applications Beyond Generic Graphs: Consideration of this framework on data types beyond traditional graphs, such as those found in natural language processing and other domains.

This research contributes a significant step towards bridging the gap between grid-based CNN capabilities and the flexibility required for graph-structured data, offering practical solutions and insights for future AI developments.