Subgraph-attentive Pooling (SubGattPool)

• Subgraph-attentive Pooling (SubGattPool) is a hierarchical graph neural network architecture that uses subgraph-level attention to capture higher-order motifs for enhanced graph classification.
• It integrates dual hierarchical attention mechanisms, with intra-hierarchy (node-level) and inter-hierarchy (level-level) operations, to aggregate multi-scale graph representations.
• Empirical evaluations on benchmarks like MUTAG and IMDB demonstrate that SubGattPool achieves competitive classification accuracy and robustness against hyperparameter variations.

Subgraph-attentive Pooling (SubGattPool) is a hierarchical graph neural network (GNN) architecture that introduces subgraph-level attention and dual hierarchical attention mechanisms for robust graph classification. Developed by Bandyopadhyay et al., SubGattPool addresses the limitations of standard neighborhood-level attention in GNNs by attending directly to sets of nodes (subgraphs) and by adaptively weighting both nodes and hierarchical graph representations (Bandyopadhyay et al., 2020).

1. Subgraph Attention: Formalism and Motivation

Standard GNN attention mechanisms focus on edges or immediate node neighborhoods, but in many real-world scenarios, higher-order node groupings (subgraphs) play a decisive role in determining graph labels. SubGattPool explicitly models such dependencies by attending over small subgraphs associated with each node.

Given an undirected attributed graph $G=(V,E,X)$ with node features $x_i \in \mathbb{R}^D$ for $v_i\in V$, a set of candidate subgraphs $S_{i\ell}$ of size at most $T$ is defined for each node. Each subgraph is a rooted induced subtree with the root at $v_i$ and depth up to $T-1$. To represent each subgraph $S_{i\ell}$ containing $s \le T$ nodes, features are ordered (e.g., via BFS), concatenated, and zero-padded: $$\hat x_{i\ell} = [\,x_{i(1)}\,\Vert\,x_{i(2)}\,\Vert\,\cdots\,\Vert\,x_{i(s)}\,\Vert\,0\,\Vert\,\cdots\,\Vert\,0]\in\mathbb{R}^{T D}.$$

$x_i \in \mathbb{R}^D$0 such subgraphs are sampled per node. The attention mechanism computes (Equation 1): $x_i \in \mathbb{R}^D$1

where $x_i \in \mathbb{R}^D$2 and $x_i \in \mathbb{R}^D$3 are shared parameters, and $x_i \in \mathbb{R}^D$4 is LeakyReLU in practice. This produces node embeddings $x_i \in \mathbb{R}^D$5 that directly incorporate information from salient higher-order motifs.

2. Hierarchical Pooling and Graph Hierarchies

To capture global graph structure and enable scalable representations, SubGattPool constructs an $x_i \in \mathbb{R}^D$6-level hierarchy, forming a sequence $x_i \in \mathbb{R}^D$7 of coarser graphs. Graph coarsening at each level uses soft-assignment matrices learned via pooling networks. The first pooling layer employs the Subgraph-attentive mechanism; subsequent levels use Graph Isomorphism Network (GIN)-based assignments. At each coarsening step: $x_i \in \mathbb{R}^D$8 where $x_i \in \mathbb{R}^D$9 and $v_i\in V$0 are adjacency and features at level $v_i\in V$1, $v_i\in V$2 is the assignment matrix, and $v_i\in V$3 are the node embeddings obtained from a GIN layer for $v_i\in V$4.

The GIN layer updates node embeddings as: $v_i\in V$5 This hierarchical construction enables SubGattPool to operate on varied graph scales, supporting the identification of discriminative graph substructures across multiple levels of granularity.

3. Hierarchical Attention Mechanisms

SubGattPool augments hierarchical pooling with two distinct attention operations for improved robustness:

3.1 Intra-hierarchy (Node-level) Attention

For any coarsened graph $v_i\in V$6 ($v_i\in V$7), intra-level attention computes importance scores for nodes: $v_i\in V$8 with $v_i\in V$9 trainable. This provides a weighted summary $S_{i\ell}$0 reflecting the node importances in $S_{i\ell}$1.

3.2 Inter-hierarchy (Level-level) Attention

To synthesize graph-level information, SubGattPool computes attention over hierarchical summaries: $S_{i\ell}$2 where $S_{i\ell}$3 is learned, yielding $S_{i\ell}$4, the final embedding for classification.

4. Network Architecture and Training

SubGattPool’s forward pass begins with subgraph-attentive embedding and pooling at the finest level, followed by a sequence of GIN embedding and pooling layers for higher levels. After intra- and inter-hierarchy attention, the resulting embedding is passed to a multi-layer perceptron (MLP) and a softmax layer to produce class scores.

The principal learnable parameters are:

• Subgraph attention: $S_{i\ell}$5, $S_{i\ell}$6,
• GIN MLP weights and biases, $S_{i\ell}$7,
• Intra-level attention vector $S_{i\ell}$8,
• Inter-level attention vector $S_{i\ell}$9,
• Classifier parameters $T$0.

The model is trained via cross-entropy loss with L2 regularization: $T$1 where $T$2 denotes all trainable weights.

5. Computational Cost and Scalability

The computational complexity for SubGattPool is dominated by:

• SubGatt layer: $T$3 for $T$4 nodes, $T$5 subgraphs per node,
• GIN layers: $T$6 per level,
• Soft-assignment pooling: $T$7, where $T$8,
• Hierarchical attentions: $T$9.

Memory requirements are $v_i$0. The architecture targets scenarios where $v_i$1, $v_i$2, $v_i$3, and $v_i$4 are moderate constants.

Component	Main Computational Cost	Main Learnable Parameters
Subgraph Attention	$v_i$5	$v_i$6
GIN Layers	$v_i$7	MLP, $v_i$8
Attention Mechanisms	$v_i$9	$T-1$0

6. Empirical Results and Ablation Studies

Experimental evaluation on seven graph classification benchmarks (MUTAG, PTC, PROTEINS, IMDB-BINARY, IMDB-MULTI, etc.) demonstrates that SubGattPool sets new state-of-the-art results on MUTAG, PTC, IMDB-B, and IMDB-M (Table 1 in (Bandyopadhyay et al., 2020)). In a synthetic clique detection task, subgraph attention reliably identifies discriminative motifs, assigning highest $T-1$1 to the true clique subgraph. Ablation studies reveal that the removal of subgraph attention or either hierarchical attention mechanism reduces class separation in the learned embeddings, as visualized with t-SNE (Figures 6–7). The model is robust to choices of $T-1$2, $T-1$3, $T-1$4, and numbers of SubGatt layers (Figure 1), indicating stable performance across varied hyperparameters.

7. Summary and Significance

SubGattPool introduces a novel subgraph-level attention mechanism, combined with hierarchical pooling and dual self-attention at both node and hierarchy levels. This architecture addresses the underrepresentation of higher-order motifs in GNNs by associating node embeddings with salient subgraphs and adaptively aggregating relevant information across multiple hierarchical graph representations. The approach yields improved or competitive classification accuracy on standard benchmarks and is empirically validated as robust to its key hyperparameters (Bandyopadhyay et al., 2020).