Can Graph Neural Networks Count Substructures? (2002.04025v4)

Abstract: The ability to detect and count certain substructures in graphs is important for solving many tasks on graph-structured data, especially in the contexts of computational chemistry and biology as well as social network analysis. Inspired by this, we propose to study the expressive power of graph neural networks (GNNs) via their ability to count attributed graph substructures, extending recent works that examine their power in graph isomorphism testing and function approximation. We distinguish between two types of substructure counting: induced-subgraph-count and subgraph-count, and establish both positive and negative answers for popular GNN architectures. Specifically, we prove that Message Passing Neural Networks (MPNNs), 2-Weisfeiler-Lehman (2-WL) and 2-Invariant Graph Networks (2-IGNs) cannot perform induced-subgraph-count of substructures consisting of 3 or more nodes, while they can perform subgraph-count of star-shaped substructures. As an intermediary step, we prove that 2-WL and 2-IGNs are equivalent in distinguishing non-isomorphic graphs, partly answering an open problem raised in Maron et al. (2019). We also prove positive results for k-WL and k-IGNs as well as negative results for k-WL with a finite number of iterations. We then conduct experiments that support the theoretical results for MPNNs and 2-IGNs. Moreover, motivated by substructure counting and inspired by Murphy et al. (2019), we propose the Local Relational Pooling model and demonstrate that it is not only effective for substructure counting but also able to achieve competitive performance on molecular prediction tasks.

Essay on Graph Neural Networks and Substructure Counting

The paper Can Graph Neural Networks Count Substructures? by Chen et al. addresses the expressive power of Graph Neural Networks (GNNs) by focusing on their ability to detect and count substructures within graphs. This approach is of paramount importance in fields like computational chemistry, biology, and social network analysis, where such capabilities of GNNs could elucidate intricate relational patterns critical to these domains.

The authors distinguish between two types of substructure counting: induced-subgraph-count and subgraph-count. They proceed to analyze the expressive power of popular GNN architectures like Message Passing Neural Networks (MPNNs), $k$-Weisfeiler-Lehman tests ($k$-WL), and Invariant Graph Networks (IGNs), through this lens. The paper critically examines the limitations of these architectures in substructure counting tasks, an approach that provides a more nuanced understanding of GNN capabilities beyond the classical graph isomorphism testing.

Central Theoretical Contributions

One of the primary contributions of the paper is demonstrating the limitations of MPNNs and $2$-IGNs in performing induced-subgraph-counts for connected substructures that consist of 3 or more nodes. This is highlighted by constructing instances of graphs that differ in their induced-subgraph-counts yet remain indistinguishable by these GNNs.

Conversely, the research provides a positive result for cases where these networks are capable of performing subgraph-counts for star-shaped patterns, thus establishing a concrete aspect of their expressive power. This is particularly significant since many real-world graph-structured data contain such star-shaped structures frequently, especially in social network analysis and communication graphs.

Furthermore, the paper extends to $k$-WL, demonstrating that with increasing $k$, these tests can count subgraphs and induced subgraphs for patterns of size $k$. However, for path patterns, they derive an upper bound on the size of the substructures that finite iterations of $k$-WL can recognize, indicating that depth and breadth constraints critically affect the network's counting ability. This finding underscores the trade-off between the depth and expressibility inherent in GNNs.

Experimental Validation and Model Innovation

Subsequent empirical evaluations reinforce the theoretical findings. They illustrate that while existing GNN architectures achieve some success on easier counting tasks (e.g., subgraph-count of $3$-stars), they struggle with more intricate patterns like triangles and chordal cycles. These results prompt the exploration of novel architectures such as Local Relational Pooling (LRP), which the authors propose as an instance-based GNN designed to count substructures effectively.

LRP shows potential by performing well on both synthetic graphs and molecular datasets, marking a promising direction for future design of GNNs that leverage local graph structures more effectively. The empirical success of LRP on diverse molecular predictions tasks like those in the QM9 and ZINC datasets further supports its utility in real-world applications.

Implications and Future Directions

The implications of this research extend to both theoretical exploration and practical application of GNNs. On the theoretical side, understanding the substructure counting capacity of GNNs potentially leads to new insights in semantic graph understanding and function approximation. Practically, the ability to count and identify specific graph substructures can enhance model performance in domains reliant on complex pattern recognition, such as drug discovery and social network dynamics.

Further research could continue to investigate the interplay between GNN architectural depth and breadth, particularly how these factors influence network expressiveness regarding more complex or larger substructures. Additionally, exploring alternative architectures that transcend the identified limitations, perhaps through hybrid or ensemble approaches, may yield even more powerful and flexible tools for graph analytics.

In conclusion, the work by Chen et al. provides a rigorous evaluation of existing GNN models concerning their substructure counting abilities, paving the way for more powerful graph analytical tools both in theory and practice. The proposal and validation of new models like LRP hold promise for future GNN advancements, especially in areas where detailed substructure detection and count prediction are pivotal.