Jumping Knowledge in Graph Neural Networks

• Jumping Knowledge is a layer-aggregation paradigm that collects intermediate node embeddings from multiple GNN layers to dynamically adjust the receptive field.
• It mitigates over-smoothing by preserving both shallow local features and deeper global context, addressing the mismatched neighborhood range problem.
• Different jump mechanisms (concatenation, max-pooling, and LSTM-based attention) offer a trade-off between computational cost and adaptive feature selection for improved task performance.

Searching arXiv for core Jumping Knowledge papers and recent theoretical follow-ups. arXiv search query: "Jumping Knowledge Networks graph" Jumping Knowledge (JK) denotes a family of graph neural network architectures that run a stack of neighborhood-aggregation layers, retain the intermediate node representations from every layer, and then combine these multi-depth representations into a final embedding instead of using only the last-layer output. In the formulation introduced by Xu et al., the final representation of node $v$ is $$\mathbf{z}_v = f\big(\mathbf{h}_v^{(1)}, \ldots, \mathbf{h}_v^{(K)}\big)$$, where each $\mathbf{h}_v^{(k)}$ summarizes information from the $k$-hop neighborhood and $f$ is a jump mechanism such as concatenation, max-pooling, or attention/LSTM-based selection (Xu et al., 2018). The central purpose of JK is to make the effective receptive field node-specific and task-dependent, thereby mitigating the performance degradation that standard deep GNNs often exhibit under over-smoothing or over-aggregation; later work has extended this perspective to application-specific systems and to the theoretical analysis of sparsified GCNs with jumping connections (Lo et al., 2022, Sun et al., 7 Jul 2025).

1. Conceptual basis

Standard neighborhood-aggregation GNNs update node representations layer by layer. After $k$ layers, the representation of a node depends on nodes up to $k$ hops away, and downstream prediction commonly uses only the deepest representation. Xu et al. identify several limitations of this design: over-smoothing or over-aggregation as depth increases, dilution of local information, graph-structure dependence of the optimal neighborhood size, and empirical performance degradation beyond roughly $2$–$3$ layers in plain GCN, GraphSAGE, and GAT settings (Xu et al., 2018).

The key observation is that the useful neighborhood range is not uniform across nodes. Nodes in dense communities may benefit from a small radius, whereas nodes in sparse regions or bridge positions may require information from farther away. JK addresses this mismatch by allowing each node to “jump” back to earlier layers and use a shallower representation when that is more informative. In the language of the original paper, JK enables “better structure-aware representation” by flexibly leveraging different neighborhood ranges for each node (Xu et al., 2018).

The original analysis also draws an analogy to random walks or diffusion. Repeated application of normalized adjacency operators causes the effective neighborhood to spread, and after many steps this process increasingly reflects global structure rather than node-specific information. This provides the intuition for why deeper layers can wash out discriminative local signals and why retaining intermediate layers can be useful (Xu et al., 2018).

A common misconception is to treat JK as merely another optimization-oriented skip connection. The published characterization is narrower and more specific: JK is related to skip connections, but its focus is adaptive neighborhood-range selection in graphs. This suggests that its main role is not only gradient stabilization but also the correction of a structural mismatch between fixed network depth and heterogeneous graph topology (Xu et al., 2018).

2. Mathematical formulation and aggregation mechanisms

For a stack of $K$ GNN layers, JK collects the layer-wise node embeddings

$$\mathbf{z}_v = f\big(\mathbf{h}_v^{(1)}, \ldots, \mathbf{h}_v^{(K)}\big)$$0

and computes the final representation through a jump function

$$\mathbf{z}_v = f\big(\mathbf{h}_v^{(1)}, \ldots, \mathbf{h}_v^{(K)}\big)$$1

The original JK paper defines three main strategies: concatenation, max-pooling across layers, and LSTM/attention-based jumping (Xu et al., 2018).

Mechanism	Definition	Stated trade-off
Concat	$$\mathbf{z}_v = f\big(\mathbf{h}_v^{(1)}, \ldots, \mathbf{h}_v^{(K)}\big)$$2	Preserves all depths; dimension grows linearly in $$\mathbf{z}_v = f\big(\mathbf{h}_v^{(1)}, \ldots, \mathbf{h}_v^{(K)}\big)$$3
MaxPool	$$\mathbf{z}_v = f\big(\mathbf{h}_v^{(1)}, \ldots, \mathbf{h}_v^{(K)}\big)$$4	Constant dimension; parameter-free hard selection
LSTM / attention	Sequence model or learned weights over $$\mathbf{z}_v = f\big(\mathbf{h}_v^{(1)}, \ldots, \mathbf{h}_v^{(K)}\big)$$5	Node-specific adaptive weighting; higher overhead

Concatenation preserves all depths equally and is often followed by a linear projection. Max-pooling performs element-wise selection across layers and can be interpreted as depth selection per node and per feature dimension. LSTM or attention variants learn content-dependent weighting over layers and therefore provide the most explicit node-specific adaptivity, at the price of additional parameters and computational cost (Xu et al., 2018).

The same formal pattern appears in later domain-specific work. In Android malware classification with graph neural networks, the adopted JK mechanism is a layer-aggregation concatenation scheme: $$\mathbf{z}_v = f\big(\mathbf{h}_v^{(1)}, \ldots, \mathbf{h}_v^{(K)}\big)$$6 followed by graph-level pooling (Lo et al., 2022). This illustrates that JK is not tied to one aggregator; rather, it is a generic layer-wise readout principle whose concrete instantiation depends on the model family and task.

3. Relationship to message passing and over-smoothing

JK is a wrapper around existing GNNs rather than a replacement for message passing. The original paper investigates JK-GCN, JK-GraphSAGE, and JK-GAT, and the Android malware study applies the same design idea to GCN, GraphSAGE, and GIN (Xu et al., 2018, Lo et al., 2022). The forward structure is consistent across these cases: compute $$\mathbf{z}_v = f\big(\mathbf{h}_v^{(1)}, \ldots, \mathbf{h}_v^{(K)}\big)$$7 standard GNN layers, collect the $$\mathbf{z}_v = f\big(\mathbf{h}_v^{(1)}, \ldots, \mathbf{h}_v^{(K)}\big)$$8 intermediate node embeddings, apply JK aggregation, and then pass the resulting embedding to the downstream classifier.

The over-smoothing problem provides the main technical motivation. In the Android study, the issue is stated directly: “GNNs are susceptible to the oversmoothing problem and their performance can degrade significantly with an increase in the number of neural network layers” (Lo et al., 2022). The intuition is that each layer mixes information from neighbors, so repeated aggregation causes node embeddings to converge to similar or nearly uniform vectors. In a linearized GCN-like view,

$$\mathbf{z}_v = f\big(\mathbf{h}_v^{(1)}, \ldots, \mathbf{h}_v^{(K)}\big)$$9

and repeated application of the mixing operator $\mathbf{h}_v^{(k)}$0 drives embeddings toward a low-rank subspace, making nodes less distinguishable (Lo et al., 2022).

JK mitigates this by preserving shallow representations that are less smoothed. Early layers retain local structural information, while deeper layers provide broader context. The final JK representation can therefore combine low-depth and high-depth information instead of being forced to rely exclusively on the deepest embedding (Lo et al., 2022). Xu et al. describe this more generally as adaptive receptive-field selection, and the Android paper describes it as a means to enable deeper $\mathbf{h}_v^{(k)}$1-layer GNNs without severe over-smoothing (Xu et al., 2018, Lo et al., 2022).

This distinction between local and global information is central. In citation, social, and protein graphs, the original JK paper argues that some nodes benefit from shallow neighborhoods and others from deeper context. In Android function call graphs, the same principle is instantiated as a balance between short call-path patterns and broader behavioral context. This suggests that JK is best understood as a multi-scale graph representation mechanism whose anti-over-smoothing effect follows from explicit access to earlier layers rather than from any change to the base aggregation rule itself (Xu et al., 2018, Lo et al., 2022).

4. Architectural integration and empirical uses

In practice, JK has been integrated into several canonical GNN backbones. The original work reports that combining the JK framework with Graph Convolutional Networks, GraphSAGE, and Graph Attention Networks consistently improves those models’ performance on social, bioinformatics, and citation networks, and demonstrates state-of-the-art results in a number of experiments (Xu et al., 2018).

A detailed application appears in Android malware classification on function call graphs (FCGs). In that setting, an FCG is defined as a directed graph $\mathbf{h}_v^{(k)}$2, where $\mathbf{h}_v^{(k)}$3 is a set of nodes representing Android API function and $\mathbf{h}_v^{(k)}$4 represents the set of inter-procedural calls. APKs are statically analyzed using Androguard to extract FCGs from DEX files, and featureless nodes are assigned structural features derived from PageRank, in/out degree, and betweenness centrality, which are standardized and stacked as node feature vectors (Lo et al., 2022).

The deployed JK architecture is explicit: input node features, a stack of $\mathbf{h}_v^{(k)}$5 GNN layers, a JK block with concatenation across all $\mathbf{h}_v^{(k)}$6 layer outputs for each node plus a linear layer, global max pooling over nodes to obtain a graph embedding, a dense layer with $\mathbf{h}_v^{(k)}$7 units and ReLU, and a softmax classifier (Lo et al., 2022). The task settings include binary malware detection on Drebin plus benign samples from AndroZoo, multiclass classification on Malnet-Tiny with four malware types plus benign, and multiclass classification over $\mathbf{h}_v^{(k)}$8 malware families on Drebin (Lo et al., 2022).

Model	Malnet-Tiny accuracy	Drebin binary accuracy
GCN-JK	89.70%	97.82%
GraphSAGE-JK	94.40%	98.00%
GIN-JK	90.00%	97.39%

On Malnet-Tiny, GraphSAGE-JK attains $\mathbf{h}_v^{(k)}$9 accuracy, weighted recall $k$0, and $k$1; GCN-JK attains $k$2 accuracy and GIN-JK $k$3 (Lo et al., 2022). The same study reports that the JK-based GCN improves the original deeper GCN performance on Malnet-Tiny from $k$4 to $k$5, a direct empirical indication that JK can rescue deeper GCNs that otherwise suffer from over-smoothing (Lo et al., 2022). On Drebin multiclass family classification, GraphSAGE-JK attains $k$6 accuracy and weighted recall $k$7, again outperforming the other JK-wrapped backbones (Lo et al., 2022).

The paper also presents UMAP visualizations of learned graph embeddings after the dense layer. These show higher class separability for GraphSAGE-JK than for GCN-JK or GIN-JK on the reported datasets. Although this is a qualitative result rather than a direct depth ablation, it is used as evidence that JK-augmented deeper models can produce more discriminative graph representations (Lo et al., 2022).

5. Theoretical developments: jumping connections and sparse effective adjacency

A recent theoretical treatment studies GCNs with jumping connections under layer-wise graph sparsification. In that work, “jumping connections” are the same basic idea as Jumping Knowledge: later layers or the final classifier directly access earlier representations rather than only the deepest embedding (Sun et al., 7 Jul 2025). The analyzed two-hidden-layer GCN has the form

$k$8

so the output is the sum of a one-layer path and a two-layer refinement path (Sun et al., 7 Jul 2025).

The theoretical picture decomposes the learned function into a base function learned by the first layer and a composite or perturbation function learned by the second layer. The concept class is written as

$k$9

where $f$0 is a sparse effective adjacency matrix that preserves the essential edges for the task (Sun et al., 7 Jul 2025). The paper’s main conclusion is that the generalization accuracy of the learned model closely approximates the highest achievable accuracy within a broad class of target functions dependent on $f$1, provided that graph sparsification preserves the essential edges that support meaningful message propagation (Sun et al., 7 Jul 2025).

The role of JK becomes especially clear in the layer-wise sparsification results. The paper shows that jumping connections lead to different sparsification requirements across layers: in a two-hidden-layer GCN, generalization is more affected by deviations of the first-layer sparsified matrix from $f$2 than by deviations in the second layer (Sun et al., 7 Jul 2025). The intuitive reason, made explicit in the analysis, is that the first-layer output reaches the classifier directly through the jump path and therefore carries most of the predictive signal, whereas the deeper path acts as a refinement term scaled by $f$3 (Sun et al., 7 Jul 2025).

This is presented as the first theoretical characterization of jumping connections’ role in sparsification requirements (Sun et al., 7 Jul 2025). A plausible implication is that JK does more than preserve information heuristically: it induces an asymmetry in how errors at different depths affect learning and generalization, thereby giving formal support to the empirical observation that early layers are often more critical than later ones in deep GCN pipelines with jumps.

6. Limitations, design trade-offs, and practical interpretation

JK introduces clear computational and modeling trade-offs. In the original paper, concatenation increases representation size proportionally to the number of layers $f$4, storing all intermediate outputs costs memory, adaptive aggregators such as LSTM add parameters and computation, and the optimal number of layers and JK strategy is dataset-dependent (Xu et al., 2018). The Android malware study likewise notes that JK concatenation increases dimensionality by a factor of $f$5, so the JK linear layer is larger, even though the reported $f$6-layer architectures remain trainable in practice (Lo et al., 2022).

The choice of JK mechanism is therefore consequential. The original experiments report that simple MaxPool JK is surprisingly effective across many datasets, while concatenation often performs well when feature dimensionality is modest, and LSTM or attention can offer slight improvements on complex datasets but may not always justify their additional complexity (Xu et al., 2018). This suggests that JK should not be identified with a single readout operator; rather, it is a design space of layer-aggregation mechanisms with different cost–adaptivity trade-offs.

Application studies also expose task-specific limits. The Android malware paper states that the deployed architecture is limited to “reasonably complex 6-layer JK networks” and does not explore deeper architectures such as DeeperGCN, edge-based GNNs such as E-GraphSAGE or E-ResGAT for network-flow-based detection, or explainable GNNs such as SubgraphX for forensic analysis (Lo et al., 2022). The paper’s recommendations are correspondingly bounded: use JK when depth is needed, combine structural node features with JK when raw semantic features are unavailable, and prefer GraphSAGE-JK for scalability in that application setting (Lo et al., 2022).

The recent theory on sparsification adds another design rule. Because generalization is more sensitive to first-layer deviations from the sparse effective adjacency $f$7 than to second-layer deviations, shallow layers should be sparsified conservatively while deeper layers can be pruned more aggressively in skip-connected GCNs with jumping connections (Sun et al., 7 Jul 2025). In practical terms, this refines the usual anti-over-smoothing narrative: JK not only preserves shallow information, but also changes which layers must preserve adjacency fidelity most carefully.

Taken together, these results position Jumping Knowledge as a general multi-depth representation principle for graphs. It is generic and plug-in, compatible with standard GCN, GraphSAGE, GAT, and GIN backbones; it addresses the structural problem that different nodes and tasks require different neighborhood ranges; and it supports both empirical gains and formal analyses of deep graph learning under sparsification and over-smoothing constraints (Xu et al., 2018, Lo et al., 2022, Sun et al., 7 Jul 2025).