Adaptive Depth Message Passing GNN

• ADMP-GNN is a graph neural network framework that dynamically adapts message passing depth based on node and graph characteristics.
• It utilizes multi-exit architectures, curvature-guided stopping, and variational techniques to tailor propagation for diverse graph regions.
• This adaptive approach mitigates common issues like oversmoothing and oversquashing, leading to improved node classification performance.

Searching arXiv for the core ADMP-GNN paper and closely related adaptive-depth/message-passing GNN work to ground the article in current literature. Adaptive Depth Message Passing GNN (ADMP-GNN) denotes a family of graph neural network methods that replace the conventional use of a single fixed message-passing depth with adaptive mechanisms that determine how far information should propagate, either globally, per graph, or per node. In the literature, the term appears most directly in “ADMP-GNN: Adaptive Depth Message Passing GNN” (Abbahaddou et al., 1 Sep 2025), which formulates adaptive depth as a multi-exit framework for node classification, but closely related work also uses curvature-guided per-vertex stopping (Hevapathige et al., 3 Mar 2025), variational depth posteriors with message filtering (Errica et al., 2023), and architecture search spaces in which effective message-passing depth emerges from selected aggregation operations (Cai et al., 2021). Across these formulations, the common premise is that fixed-depth message passing is suboptimal because different graph regions and tasks exhibit different propagation requirements.

1. Conceptual definition and scope

ADMP-GNN is motivated by the observation that standard message-passing GNNs apply the same number of propagation steps to all nodes, despite substantial variation in local neighborhood density, centrality, structural position, and the amount of context needed for accurate prediction (Abbahaddou et al., 1 Sep 2025). The core claim is that the “optimal number of message-passing layers varies for nodes with different characteristics” (Abbahaddou et al., 1 Sep 2025). This departs from the standard practice of selecting a single depth by grid search and applying it uniformly.

The main literature distinguishes several forms of adaptivity. In the multi-exit formulation of ADMP-GNN, predictions are produced at every layer and a node-specific exit layer is selected by a structural policy (Abbahaddou et al., 1 Sep 2025). In curvature-guided models, a stopping depth $T(x)$ is assigned to each vertex based on learnable Bakry–Émery curvature, with high-curvature vertices using shallower propagation and low-curvature vertices deeper propagation (Hevapathige et al., 3 Mar 2025). In variational adaptive message passing, depth is treated as a latent random variable $L$, and message flow is further modulated by learned soft filters $\mathbf F$ (Errica et al., 2023). In message-passing NAS, the effective depth is not preset but emerges from which searched cells actually contain neighbor aggregation (Cai et al., 2021).

A plausible implication is that ADMP-GNN is best understood not as a single architecture, but as a design principle for message-passing systems in which depth becomes a learned or inferred property rather than a fixed hyperparameter.

2. Motivation: why fixed depth is inadequate

The principal empirical motivation comes from the finding that shallow and deep propagation are advantageous for different nodes or graph regions. “ADMP-GNN: Adaptive Depth Message Passing GNN” constructs synthetic graphs by merging sparse and dense subgraphs extracted from Computers and Photo, then trains GCNs with depths from 0 to 10 and evaluates the sparse and dense parts separately (Abbahaddou et al., 1 Sep 2025). The reported outcome is that the best depth differs between sparse and dense regions, with dense subgraphs degrading more quickly as depth increases and sparse subgraphs benefiting from more propagation before accuracy drops (Abbahaddou et al., 1 Sep 2025). The same paper further reports oracle accuracies obtained by choosing the best layer independently for each test node, and these oracle values are substantially higher than any fixed-depth model, indicating considerable headroom for node-specific exit selection (Abbahaddou et al., 1 Sep 2025).

Related work explains this inadequacy in terms of classical pathologies of deep message passing. “Adaptive Message Passing: A General Framework to Mitigate Oversmoothing, Oversquashing, and Underreaching” identifies three failure modes: underreaching in shallow networks, oversmoothing under repeated averaging, and oversquashing when many distant signals are compressed into fixed-size representations (Errica et al., 2023). “Rethinking Graph Neural Architecture Search from Message-passing” adds that optimal depth is dataset-dependent and that deeper propagation can worsen performance because of over-smoothing, so manual enumeration of depth is inherently restrictive (Cai et al., 2021).

Theoretical analyses in later adaptive-depth work sharpen this point. Curvature-based depth adaptation establishes a bound linking local mixing time to curvature,

$$\tau_x(\epsilon) \le \frac{\log(1/\epsilon)}{\kappa(x)},$$

which is used to argue that high-curvature vertices mix rapidly and therefore require fewer layers, while low-curvature vertices benefit from deeper propagation (Hevapathige et al., 3 Mar 2025). A separate homophily-sensitive theory defines a local signal preservation factor

$\alpha_v=\frac{1+d_v^+-d_v^-}{d_v+1},$

and shows that the usefulness of aggregation depends jointly on same-label neighbors, opposite-label neighbors, and degree, so a single depth can over-aggregate some nodes and under-aggregate others (Hevapathige et al., 10 Nov 2025).

3. Core architectural patterns

The direct ADMP-GNN formulation in (Abbahaddou et al., 1 Sep 2025) is a model-agnostic wrapper over standard message-passing GNNs. The underlying message passing follows

$$m^{(\ell)}_v = \psi^{(\ell)}(\{h_u^{(\ell-1)} : u \in \mathcal{N}(v)\}), \qquad h^{(\ell)}_v = \phi^{(\ell)}(h_v^{(\ell-1)}, m_v^{(\ell)}),$$

with $h_v^{(0)}=x_v$ (Abbahaddou et al., 1 Sep 2025). The distinctive addition is a prediction head at every layer: $p^{(\ell)}_v = \phi^{(\ell)}_{\mathbf{Ex}(h_v^{(\ell-1)}, m_v^{(\ell)}) = \text{Softmax}(\widetilde{W}^{(\ell)} m_v^{(\ell)}),$ together with a continuation update

$h_v^{(\ell+1)} = \phi^{(\ell)}_{\mathbf{Ct}(h_v^{(\ell-1)}, m_v^{(\ell)}),$

and a layer-0 classifier

$$p_v^{(0)} = \text{Softmax}(\widetilde{W}^{(0)} x_v)$$

(Abbahaddou et al., 1 Sep 2025). This yields a sequence of layer-wise classifiers from depth $L$0 through $L$1, after which each node is assigned an exit layer.

Other ADMP-GNN variants implement adaptivity differently. In curvature-guided adaptive depth, the architecture remains a standard GNN backbone such as GCN, GAT, GraphSAGE, GIN, SGC, MixHop, or DIR-GNN, but a curvature estimation module computes $L$2 per vertex, converts it to a stopping depth $L$3, and runs adaptive message passing in which neighbors contribute either their current-layer representation or their final stopped representation (Hevapathige et al., 3 Mar 2025). The update is

$L$4

for $L$5, and prediction uses $L$6 (Hevapathige et al., 3 Mar 2025).

In the variational AMP framework, adaptive depth is coupled with soft message filtering. The filtered update is

$L$7

where $L$8 modulates outgoing messages from node $L$9 at layer $\mathbf F$0 (Errica et al., 2023). This makes adaptation act simultaneously on depth and message content.

In differentiable architecture search, the Graph Neural Architecture Paradigm (GAP) defines a directed tree with feature filtering before aggregation, one neighbor-aggregation stage per branch, and feature filtering afterward (Cai et al., 2021). In this setting, message-passing depth is “the number of stacked graph architectures that contain neighbor aggregation,” so depth is determined by the selected operations rather than imposed externally (Cai et al., 2021).

4. Depth selection mechanisms

ADMP-GNN methods differ most sharply in how they decide the stopping depth.

In the multi-exit ADMP-GNN of (Abbahaddou et al., 1 Sep 2025), depth selection is not learned through an internal halting controller at inference time. Instead, the model assumes that structurally similar nodes should share the same best exit layer. Structural similarity is approximated using centrality metrics—degree, $\mathbf F$1-core, PageRank, and Walk Count for length 2—after which nodes are sorted by centrality, divided into $\mathbf F$2 equal-sized clusters, and validation nodes are used to select the best layer for each cluster: $\mathbf F$3 Test nodes in cluster $\mathbf F$4 then use exit layer $\mathbf F$5 (Abbahaddou et al., 1 Sep 2025). The paper argues that validation nodes are preferable to training nodes for this policy because training accuracy is overly optimistic across layers (Abbahaddou et al., 1 Sep 2025).

The curvature-based method of (Hevapathige et al., 3 Mar 2025) uses a rank-based depth assignment rule: $\mathbf F$6 Here $\mathbf F$7 is a threshold hyperparameter. Smaller $\mathbf F$8 tends to allow deeper propagation for more vertices, whereas larger $\mathbf F$9 yields earlier stopping (Hevapathige et al., 3 Mar 2025). The theoretical basis is a curvature-dependent exponential decay of feature distinctiveness,

$$\tau_x(\epsilon) \le \frac{\log(1/\epsilon)}{\kappa(x)},$$0

which motivates smaller $$\tau_x(\epsilon) \le \frac{\log(1/\epsilon)}{\kappa(x)},$$1 for larger $$\tau_x(\epsilon) \le \frac{\log(1/\epsilon)}{\kappa(x)},$$2 (Hevapathige et al., 3 Mar 2025).

The variational AMP framework selects depth by learning a posterior $$\tau_x(\epsilon) \le \frac{\log(1/\epsilon)}{\kappa(x)},$$3 over possible depths rather than committing to a single integer depth (Errica et al., 2023). The support of $$\tau_x(\epsilon) \le \frac{\log(1/\epsilon)}{\kappa(x)},$$4 is truncated at a high quantile such as $$\tau_x(\epsilon) \le \frac{\log(1/\epsilon)}{\kappa(x)},$$5, and prediction averages outputs across all supported depths with weights determined by the learned posterior: $$\tau_x(\epsilon) \le \frac{\log(1/\epsilon)}{\kappa(x)},$$6 (Errica et al., 2023). Depth adaptation here is distributional rather than node-wise.

The message-passing NAS perspective in (Cai et al., 2021) yields another mechanism: initialize with a depth related to graph diameter, search a super-network, derive a discrete subarchitecture, count the number of cells that actually contain neighbor aggregation, and use that count as the next depth until convergence. In the reported algorithm, $$\tau_x(\epsilon) \le \frac{\log(1/\epsilon)}{\kappa(x)},$$7 is initialized as half of the average graph diameter and then updated to the number of cells with at least one neighbor aggregation in the derived network (Cai et al., 2021). This makes depth an emergent property of the architecture search process.

5. Training formulations and optimization

The direct ADMP-GNN paper introduces two training paradigms for multi-exit models (Abbahaddou et al., 1 Sep 2025). Aggregate Loss Minimization (ALM) jointly minimizes the sum of losses across all exits: $$\tau_x(\epsilon) \le \frac{\log(1/\epsilon)}{\kappa(x)},$$8 with $$\tau_x(\epsilon) \le \frac{\log(1/\epsilon)}{\kappa(x)},$$9 equal to cross-entropy (Abbahaddou et al., 1 Sep 2025). The paper reports that this simple joint objective suffers from gradient conflicts, especially because early layers are influenced by many deeper losses (Abbahaddou et al., 1 Sep 2025).

Sequential Training (ST) is the preferred alternative. It trains layer $\alpha_v=\frac{1+d_v^+-d_v^-}{d_v+1},$0, freezes it, then trains each deeper layer after previous layers are fixed, using the recursion

$\alpha_v=\frac{1+d_v^+-d_v^-}{d_v+1},$1

with

$\alpha_v=\frac{1+d_v^+-d_v^-}{d_v+1},$2

(Abbahaddou et al., 1 Sep 2025). The reported trade-off is that ST involves $\alpha_v=\frac{1+d_v^+-d_v^-}{d_v+1},$3 training phases and thus higher wall-clock time, but yields better multi-exit behavior and greater stability (Abbahaddou et al., 1 Sep 2025).

Curvature-guided adaptive depth augments the task loss with curvature regularization: $\alpha_v=\frac{1+d_v^+-d_v^-}{d_v+1},$4 where

$\alpha_v=\frac{1+d_v^+-d_v^-}{d_v+1},$5

and $\alpha_v=\frac{1+d_v^+-d_v^-}{d_v+1},$6 in experiments (Hevapathige et al., 3 Mar 2025). The finite family $\alpha_v=\frac{1+d_v^+-d_v^-}{d_v+1},$7 of MLP-generated functions approximates the otherwise intractable curvature optimization, and $\alpha_v=\frac{1+d_v^+-d_v^-}{d_v+1},$8 is small in experiments, for example $\alpha_v=\frac{1+d_v^+-d_v^-}{d_v+1},$9 (Hevapathige et al., 3 Mar 2025).

The AMP framework uses variational inference and optimizes an ELBO over latent parameters $$m^{(\ell)}_v = \psi^{(\ell)}(\{h_u^{(\ell-1)} : u \in \mathcal{N}(v)\}), \qquad h^{(\ell)}_v = \phi^{(\ell)}(h_v^{(\ell-1)}, m_v^{(\ell)}),$$0, depth $$m^{(\ell)}_v = \psi^{(\ell)}(\{h_u^{(\ell-1)} : u \in \mathcal{N}(v)\}), \qquad h^{(\ell)}_v = \phi^{(\ell)}(h_v^{(\ell-1)}, m_v^{(\ell)}),$$1, and message filters $$m^{(\ell)}_v = \psi^{(\ell)}(\{h_u^{(\ell-1)} : u \in \mathcal{N}(v)\}), \qquad h^{(\ell)}_v = \phi^{(\ell)}(h_v^{(\ell-1)}, m_v^{(\ell)}),$$2 (Errica et al., 2023). After factorization, the practical objective becomes a finite sum over supported depths,

$$m^{(\ell)}_v = \psi^{(\ell)}(\{h_u^{(\ell-1)} : u \in \mathcal{N}(v)\}), \qquad h^{(\ell)}_v = \phi^{(\ell)}(h_v^{(\ell-1)}, m_v^{(\ell)}),$$3

(Errica et al., 2023). This formalism treats all depths in the supported range as active contributors during training.

The differentiable NAS formulation in (Cai et al., 2021) adopts a DARTS-like relaxation,

$$m^{(\ell)}_v = \psi^{(\ell)}(\{h_u^{(\ell-1)} : u \in \mathcal{N}(v)\}), \qquad h^{(\ell)}_v = \phi^{(\ell)}(h_v^{(\ell-1)}, m_v^{(\ell)}),$$4

jointly optimizing network weights $$m^{(\ell)}_v = \psi^{(\ell)}(\{h_u^{(\ell-1)} : u \in \mathcal{N}(v)\}), \qquad h^{(\ell)}_v = \phi^{(\ell)}(h_v^{(\ell-1)}, m_v^{(\ell)}),$$5 and architecture parameters $$m^{(\ell)}_v = \psi^{(\ell)}(\{h_u^{(\ell-1)} : u \in \mathcal{N}(v)\}), \qquad h^{(\ell)}_v = \phi^{(\ell)}(h_v^{(\ell-1)}, m_v^{(\ell)}),$$6, then recovering a discrete architecture by

$$m^{(\ell)}_v = \psi^{(\ell)}(\{h_u^{(\ell-1)} : u \in \mathcal{N}(v)\}), \qquad h^{(\ell)}_v = \phi^{(\ell)}(h_v^{(\ell-1)}, m_v^{(\ell)}),$$7

Only the strongest non-zero operation is retained per edge, and only the strongest incoming edge per node is kept (Cai et al., 2021).

6. Empirical findings and comparative picture

The direct ADMP-GNN evaluation focuses on node classification over thirteen benchmarks including Cora, CiteSeer, PubMed, CS, ogbn-arxiv, Genius, Amazon Computers, Amazon Photo, Chameleon, Squirrel, Cornell, Texas, and Wisconsin, with the main reported tables emphasizing six datasets: Cora, CiteSeer, CS, PubMed, Genius, and ogbn-arxiv (Abbahaddou et al., 1 Sep 2025). Baselines include GCN, GIN, JKNet variants, GCNII / Residuals, AdaGCN, and GPR-GNN, and baseline depth is grid-searched from 0 to 5 for fairness (Abbahaddou et al., 1 Sep 2025). The main evaluation metric is classification accuracy as mean ± standard deviation over 10 runs, using Adam and an NVIDIA A100, with maximum depth fixed to $$m^{(\ell)}_v = \psi^{(\ell)}(\{h_u^{(\ell-1)} : u \in \mathcal{N}(v)\}), \qquad h^{(\ell)}_v = \phi^{(\ell)}(h_v^{(\ell-1)}, m_v^{(\ell)}),$$8 for ADMP-GNN in the principal comparisons (Abbahaddou et al., 1 Sep 2025).

For the GCN backbone, the paper reports the following accuracies for ADMP-GNN versus GCN: Cora 81.22 vs 80.78, CiteSeer 71.33 vs 71.25, CS 91.49 vs 92.20, PubMed 79.02 vs 79.32, Genius 80.17 vs 80.76, and ogbn-arxiv 69.29 vs 64.37 (Abbahaddou et al., 1 Sep 2025). For the GIN backbone, ADMP-GIN is reported as 78.07 vs 77.73 on Cora, 65.41 vs 65.23 on CiteSeer, 90.82 vs 90.29 on CS, 76.46 vs 76.05 on PubMed, and 60.85 vs 60.70 on ogbn-arxiv (Abbahaddou et al., 1 Sep 2025). The same paper states that Sequential Training consistently outperforms Aggregate Loss Minimization and that oracle accuracies are much higher than any fixed-depth model, with GCN oracle values of 89.43 on Cora, 81.96 on CiteSeer, 97.24 on CS, 90.13 on PubMed, 85.97 on Genius, and 79.64 on ogbn-arxiv (Abbahaddou et al., 1 Sep 2025).

The curvature-guided method reports evaluation on 21 datasets spanning vertex classification, vertex regression, graph classification, and graph regression, and states consistent gains over GCN, GAT, GraphSAGE, SGC, MixHop, and DIR-GNN on homophilic graphs, heterophilic graphs, and large-scale ogbn-arxiv (Hevapathige et al., 3 Mar 2025). It further reports that improvements are often larger on heterophilic graphs, that oversmoothing robustness persists up to 64 layers, and that replacing learnable Bakry–Émery curvature with Ollivier-Ricci, Forman-Ricci, Jost–Liu curvature, effective resistance, betweenness centrality, or degree performs worse overall (Hevapathige et al., 3 Mar 2025).

The variational AMP framework evaluates five long-range datasets—Diameter, SSSP, Eccentricity, peptides-func, and peptides-struct—and reports average test-error reductions of 63% on Diameter, 72% on SSSP, and 32% on Eccentricity relative to base architectures (Errica et al., 2023). On chemical tasks, it reports improvements of about 2–3.4% AP on peptides-func and state-of-the-art-level MAE on peptides-struct (Errica et al., 2023). These results address a broader long-range setting than node classification alone.

In message-passing NAS, GNAS is evaluated on ZINC, PATTERN, CLUSTER, MNIST, and CIFAR10 and is reported to outperform manual GNNs such as GCN, GIN, GraphSAGE, GAT, GatedGCN, and MoNet, as well as the search-based baseline GraphNAS, while using much less search time (Cai et al., 2021). On ZINC, the searched depth reportedly converges to around 12–14 regardless of the initial depth when the initialization is not too small, supporting the claim that effective depth can be learned rather than preset (Cai et al., 2021).

7. Interpretations, limitations, and relation to adjacent work

A central misconception is that adaptive depth necessarily means a fully learned per-node halting policy inside the network. The direct ADMP-GNN formulation explicitly does not use “a learned per-node halting gate inside the network at inference time”; instead, it combines multi-layer exits, a training scheme to make each layer a strong predictor, and a heuristic policy based on structural similarity (Abbahaddou et al., 1 Sep 2025). By contrast, curvature-guided stopping is more intrinsically node-wise, while variational AMP learns a posterior over depths rather than a hard node-specific halting layer (Hevapathige et al., 3 Mar 2025, Errica et al., 2023). The term therefore covers multiple adaptation mechanisms rather than one canonical controller.

Another misconception is that adaptive depth and message filtering are interchangeable. The literature suggests they are related but distinct. In (Errica et al., 2023), depth adaptation addresses underreaching by learning how many layers are needed, while message filtering suppresses irrelevant or harmful information to mitigate oversmoothing and oversquashing. In (Cai et al., 2021), the analogous decomposition is between feature filtering and neighbor aggregation, with the former controlling what is sent or preserved and the latter determining the actual message-passing step. This suggests that adaptive-depth message passing is often most effective when depth control is coupled with some form of content control.

The main limitations are method-specific. The direct ADMP-GNN paper assumes that structural similarity correlates with optimal depth, relies on validation-node-driven cluster policies, and incurs heavier training cost under Sequential Training; it is also evaluated mainly on node classification (Abbahaddou et al., 1 Sep 2025). The curvature-based approach approximates curvature using a finite family of MLP-generated functions and therefore computes an upper bound on the true curvature rather than an exact quantity (Hevapathige et al., 3 Mar 2025). The AMP framework introduces additional variational machinery and probabilistic assumptions, including bounded-support depth distributions and deterministic Dirac message-filter posteriors (Errica et al., 2023). The homophily-sensitive theory in (Hevapathige et al., 10 Nov 2025) is derived under a simplified CSBM with binary labels and row-normalized averaging, omitting symmetric normalization, learned weights, and nonlinearities.

Taken together, these works position ADMP-GNN as a broad research direction at the intersection of dynamic computation, long-range graph representation learning, and message-passing control. The unifying idea is that depth should not be treated as a universal constant. Instead, it can be assigned by structural heuristics (Abbahaddou et al., 1 Sep 2025), geometric quantities such as Bakry–Émery curvature (Hevapathige et al., 3 Mar 2025), latent-variable inference over depth distributions and filters (Errica et al., 2023), or architecture search over message-passing primitives (Cai et al., 2021). A plausible implication is that future ADMP-GNN research will continue to move from manually chosen propagation horizons toward data-dependent, theoretically informed depth policies that operate at finer granularity than the graph-wide hyperparameter choices of conventional GNNs.