Efficient Higher-order Subgraph Attribution via Message Passing

Abstract: Explaining graph neural networks (GNNs) has become more and more important recently. Higher-order interpretation schemes, such as GNN-LRP (layer-wise relevance propagation for GNN), emerged as powerful tools for unraveling how different features interact thereby contributing to explaining GNNs. GNN-LRP gives a relevance attribution of walks between nodes at each layer, and the subgraph attribution is expressed as a sum over exponentially many such walks. In this work, we demonstrate that such exponential complexity can be avoided. In particular, we propose novel algorithms that enable to attribute subgraphs with GNN-LRP in linear-time (w.r.t. the network depth). Our algorithms are derived via message passing techniques that make use of the distributive property, thereby directly computing quantities for higher-order explanations. We further adapt our efficient algorithms to compute a generalization of subgraph attributions that also takes into account the neighboring graph features. Experimental results show the significant acceleration of the proposed algorithms and demonstrate the high usefulness and scalability of our novel generalized subgraph attribution method.

• The paper introduces sGNN-LRP, an exact sum-product message-passing algorithm that reduces subgraph attribution from exponential walk enumeration to O(L|S|²) complexity in network depth L.
• The method achieves speedups of up to four orders of magnitude, processing cases where naive GNN-LRP runs out of memory or exceeds time limits, including REDDIT-B and Graph-SST2 benchmarks.
• The paper proposes a discount-based attribution that accounts for walks leaving a subgraph and often improves activation and pruning evaluations on real datasets, although selecting its α parameter remains task-dependent.

Overview

GNN-LRP (layer-wise relevance propagation for graph neural networks) attributes a GNN's prediction to individual walks—ordered node sequences traversed layer by layer—and defines the relevance of a subgraph as the sum of relevances over all walks that remain inside it. Because the number of such walks grows as $\mathcal{O}(|\mathcal{S}|^L)$ in network depth $L$, subgraph attribution via naive summation was previously restricted to small graphs and shallow networks. This paper, by Xiong et al., removes that bottleneck. The authors derive a new propagation rule, sGNN-LRP, that computes the exact subgraph relevance in a single backward pass with complexity $O(L|\mathcal{S}|^2)$, and they introduce a generalized attribution that discounts walks stepping outside the subgraph (2605.22385).

The methodological contribution is as notable as the computational one: sGNN-LRP is not obtained by heuristic modification of an existing rule but is derived systematically as a sum-product (belief propagation) message passing algorithm for an explicitly defined target quantity. The authors show this procedure also recovers standard LRP and original GNN-LRP rules, positioning it as a general framework for deriving LRP-type propagation rules.

Background: GNN-LRP and its exponential cost

GNN-LRP applies higher-order deep Taylor decomposition to a GNN unfolded into a feed-forward network whose "neurons" are node-feature pairs. For a GCN-style combine step $$\mathcal{C}^{(l)}(\mathbf{Z}) = \text{ReLU}(\mathbf{Z}\mathbf{W}^{(l)})$$, relevance is propagated along a walk $\mathbf{m}$ via

$$\breve{\mathbf{r}}^{(l, m_l)} = \mathbf{T}^{l, m_l, m_{l+1}} \breve{\mathbf{r}}^{(l+1, m_{l+1})},$$

where the propagation matrix entries are $$T_{n,n'}^{l,m,m'} = \Lambda_{m,m'} H_{m,n}^{(l)} W^{\uparrow(l)}_{n,n'} / \sum_{m'',n''}\Lambda_{m'',m'} H_{m'',n''}^{(l)} W^{\uparrow(l)}_{n'',n'}$$, with $\mathbf{W}^\uparrow$ chosen per the selected LRP variant ($\epsilon$, $\gamma$, $L$0). Subgraph relevance is then defined as $L$1. Evaluating this sum naively requires enumerating all walks confined to $L$2, which is exponential in depth; the authors note this limitation afflicts higher-order attribution methods generally, since $L$3-th order attribution over $L$4 features involves $L$5 combinations.

LRP as message passing

The central theoretical observation is that the relevance of a neuron-level walk has exactly the decomposable structure of a Markov chain joint distribution:

$L$6

Interpreting propagation matrices as transition probabilities (formally—they may be negative or unnormalized) and top-layer relevances as initial beliefs, any sum of walk relevances corresponds to a marginal probability computable by the sum-product algorithm. Applying this to the subgraph target quantity yields the main result:

Theorem 1 (informal). Running the rule

$L$7

backward from $L$8 to $L$9 yields messages equal to partial-walk relevances restricted to $O(L|\mathcal{S}|^2)$0, so that $O(L|\mathcal{S}|^2)$1 after one pass.

The only change relative to the original GNN-LRP rule is an additional sum pooling messages over subgraph nodes at each step—yet this collapses the exponential walk enumeration into linear-in-depth computation. The proof assumes normalized propagation matrices purely for expositional clarity; unnormalized matrices merely rescale messages by layer-dependent constants, which the authors state are practically irrelevant.

The paper further shows the framework's generality by re-deriving standard LRP for feed-forward networks and the original GNN-LRP walk-relevance rule as message passing for their respective target quantities, and provides equivalent formulations in the notation of Schnake et al.'s original GNN-LRP paper.

Forward-hook implementation

Beyond direct backward message passing, the authors prove that a modified forward-hook trick computes $O(L|\mathcal{S}|^2)$2 via automatic differentiation. The combine step is rewritten with detached terms and a constant mask matrix $O(L|\mathcal{S}|^2)$3 (ones on subgraph columns, zeros elsewhere); then

$O(L|\mathcal{S}|^2)$4

This yields a simpler, faster, and less memory-intensive implementation than explicit message passing, requiring no custom backward code beyond standard autograd.

Generalized subgraph attribution

The original definition ignores any walk that leaves $O(L|\mathcal{S}|^2)$5 even once, capturing only the property that predictions on $O(L|\mathcal{S}|^2)$6 and $O(L|\mathcal{S}|^2)$7 agree. The authors argue a good subgraph attribution should also satisfy the complementary property—that predictions on $O(L|\mathcal{S}|^2)$8 diverge from those on $O(L|\mathcal{S}|^2)$9—and propose a discounted variant:

$$\mathcal{C}^{(l)}(\mathbf{Z}) = \text{ReLU}(\mathbf{Z}\mathbf{W}^{(l)})$$0

with walks entirely outside $$\mathcal{C}^{(l)}(\mathbf{Z}) = \text{ReLU}(\mathbf{Z}\mathbf{W}^{(l)})$$1 receiving zero weight. At $$\mathcal{C}^{(l)}(\mathbf{Z}) = \text{ReLU}(\mathbf{Z}\mathbf{W}^{(l)})$$2 this recovers the original definition. Computation exploits the decomposition $$\mathcal{C}^{(l)}(\mathbf{Z}) = \text{ReLU}(\mathbf{Z}\mathbf{W}^{(l)})$$3: the first term follows from a message passing rule that multiplies each transition by $$\mathcal{C}^{(l)}(\mathbf{Z}) = \text{ReLU}(\mathbf{Z}\mathbf{W}^{(l)})$$4, and the second is an ordinary sGNN-LRP run on the complement. A soft-masked forward-hook variant (mask value $$\mathcal{C}^{(l)}(\mathbf{Z}) = \text{ReLU}(\mathbf{Z}\mathbf{W}^{(l)})$$5 outside $$\mathcal{C}^{(l)}(\mathbf{Z}) = \text{ReLU}(\mathbf{Z}\mathbf{W}^{(l)})$$6) is likewise proven correct.

Computational efficiency results

Experiments on BA-2motif, MUTAG, Mutagenicity, REDDIT-BINARY, and Graph-SST2 (GIN models of depth 3–7, plus a GCN-3) quantify the speedup. On BA-2motif with $$\mathcal{C}^{(l)}(\mathbf{Z}) = \text{ReLU}(\mathbf{Z}\mathbf{W}^{(l)})$$7, naive GNN-LRP scales exponentially in depth—from 224 ms at $$\mathcal{C}^{(l)}(\mathbf{Z}) = \text{ReLU}(\mathbf{Z}\mathbf{W}^{(l)})$$8 to $$\mathcal{C}^{(l)}(\mathbf{Z}) = \text{ReLU}(\mathbf{Z}\mathbf{W}^{(l)})$$9 ms at $\mathbf{m}$0 and $\mathbf{m}$1 ms at $\mathbf{m}$2—while sGNN-LRP takes 4.22, 6.44, and 9.81 ms respectively. Naive GNN-LRP fails outright (out-of-memory or exceeding a 360 s budget) on REDDIT-B with GIN-5 and requires $\mathbf{m}$3 ms on Graph-SST2, where sGNN-LRP needs 29.94 ms. Representative timings:

Dataset	Model	Naive GNN-LRP	sGNN-LRP
BA-2motif	GIN-3	224.22 ms	4.22 ms
BA-2motif	GIN-7	$\mathbf{m}$4 ms	9.81 ms
MUTAG	GIN-3	$\mathbf{m}$5 ms	28.90 ms
REDDIT-B	GIN-5	failed	195.43 ms
Graph-SST2	GCN-3	$\mathbf{m}$6 ms	29.94 ms

Against other explainers, sGNN-LRP is substantially faster than GNNExplainer and comparable to gradient heatmaps and (Grad-)CAM. Since its cost matches a single forward/backward pass, subgraph attribution becomes feasible for deeper networks and larger graphs at roughly the cost of prediction itself—an implication that directly enables the deeper-walk analyses previously out of reach.

Evaluation of the generalized attribution

Node ordering quality was assessed in activation mode (greedily maximizing accumulated subgraph relevance) and pruning mode (greedily minimizing deviation from full-graph relevance), evaluated by AUAC and AUPC respectively. On synthetic BA-2motif, where out-of-motif nodes carry no signal, performance peaks near $\mathbf{m}$7, confirming the original definition is nearly optimal there. On MUTAG and Graph-SST2, however, the optimal $\mathbf{m}$8 is consistently nonzero for both activation and pruning tasks, indicating the original strict definition is not universally best. Assuming per-method tuning of $\mathbf{m}$9, sGNN-LRP outperforms GNNExplainer, gradient heatmaps, and (Grad-)CAM in 6 of 8 activation/pruning settings. The authors note that AUAC/AUPC differ markedly between positive and negative samples, which they attribute to unequal model predictive capability across classes—a caveat when interpreting aggregate scores.

Limitations and open questions

Several constraints qualify the results. The theoretical analysis restricts itself to normalized propagation matrices, with the unnormalized case handled only informally. The greedy node-ordering procedures are approximations to infeasible combinatorial optimizations, so AUAC/AUPC reflect ordering heuristics rather than optimal subsets. The choice of discounting parameter $$\breve{\mathbf{r}}^{(l, m_l)} = \mathbf{T}^{l, m_l, m_{l+1}} \breve{\mathbf{r}}^{(l+1, m_{l+1})},$$0 is task-dependent and currently must be tuned empirically; the authors explicitly defer principled tuning procedures to future work. Evaluation of explanation quality relies on surrogate tasks (activation/pruning correlation with model output) except on BA-2motif, leaving open how well the generalized attribution aligns with ground-truth explanations on real datasets where motifs are unknown. Finally, empirical validation covers GIN and GCN architectures; extension of the derived rules to other GNN families inherits from Schnake et al. rather than being demonstrated here.

Conclusion

This paper reformulates GNN-LRP subgraph attribution as sum-product message passing, yielding an exact single-pass algorithm whose $$\breve{\mathbf{r}}^{(l, m_l)} = \mathbf{T}^{l, m_l, m_{l+1}} \breve{\mathbf{r}}^{(l+1, m_{l+1})},$$1 complexity replaces the previous exponential dependence on network depth, with measured speedups of up to four orders of magnitude and applicability where the naive method fails entirely. The same derivation framework produces a generalized, discount-weighted subgraph relevance that improves node-ordering performance on real datasets, and it uniformly recovers existing LRP rules as special cases. The principal open issues are principled selection of the discount parameter and validation against ground-truth explanations beyond synthetic benchmarks.