---
title: Adaptive Depth Message Passing GNN
url: https://www.emergentmind.com/topics/adaptive-depth-message-passing-gnn-admp-gnn
type: topic
---

# Adaptive Depth Message Passing GNN

Searching arXiv for the core ADMP-GNN paper and closely related adaptive-depth/message-passing GNN work to ground the article in current literature.
Adaptive Depth Message Passing GNN (ADMP-GNN) denotes a family of graph neural network methods that replace the conventional use of a single fixed message-passing depth with adaptive mechanisms that determine how far information should propagate, either globally, per graph, or per node. In the literature, the term appears most directly in “ADMP-GNN: Adaptive Depth Message Passing GNN” [2509.01170], which formulates adaptive depth as a multi-exit framework for node classification, but closely related work also uses curvature-guided per-vertex stopping [2503.01079], variational depth posteriors with message filtering [2312.16560], and architecture search spaces in which effective message-passing depth emerges from selected aggregation operations [2103.14282]. Across these formulations, the common premise is that fixed-depth message passing is suboptimal because different graph regions and tasks exhibit different propagation requirements.

## 1. Conceptual definition and scope

ADMP-GNN is motivated by the observation that standard message-passing GNNs apply the same number of propagation steps to all nodes, despite substantial variation in local neighborhood density, centrality, structural position, and the amount of context needed for accurate prediction [2509.01170]. The core claim is that the “optimal number of message-passing layers varies for nodes with different characteristics” [2509.01170]. This departs from the standard practice of selecting a single depth by grid search and applying it uniformly.

The main literature distinguishes several forms of adaptivity. In the multi-exit formulation of ADMP-GNN, predictions are produced at every layer and a node-specific exit layer is selected by a structural policy [2509.01170]. In curvature-guided models, a stopping depth \(T(x)\) is assigned to each vertex based on learnable Bakry–Émery curvature, with high-curvature vertices using shallower propagation and low-curvature vertices deeper propagation [2503.01079]. In variational adaptive message passing, depth is treated as a latent random variable \(L\), and message flow is further modulated by learned soft filters \(\mathbf F\) [2312.16560]. In message-passing NAS, the effective depth is not preset but emerges from which searched cells actually contain neighbor aggregation [2103.14282].

A plausible implication is that ADMP-GNN is best understood not as a single architecture, but as a design principle for message-passing systems in which depth becomes a learned or inferred property rather than a fixed hyperparameter.

## 2. Motivation: why fixed depth is inadequate

The principal empirical motivation comes from the finding that shallow and deep propagation are advantageous for different nodes or graph regions. “ADMP-GNN: Adaptive Depth Message Passing GNN” constructs synthetic graphs by merging sparse and dense subgraphs extracted from Computers and Photo, then trains GCNs with depths from 0 to 10 and evaluates the sparse and dense parts separately [2509.01170]. The reported outcome is that the best depth differs between sparse and dense regions, with dense subgraphs degrading more quickly as depth increases and sparse subgraphs benefiting from more propagation before accuracy drops [2509.01170]. The same paper further reports oracle accuracies obtained by choosing the best layer independently for each test node, and these oracle values are substantially higher than any fixed-depth model, indicating considerable headroom for node-specific exit selection [2509.01170].

Related work explains this inadequacy in terms of classical pathologies of deep message passing. “Adaptive Message Passing: A General Framework to Mitigate Oversmoothing, Oversquashing, and Underreaching” identifies three failure modes: underreaching in shallow networks, oversmoothing under repeated averaging, and oversquashing when many distant signals are compressed into fixed-size representations [2312.16560]. “Rethinking Graph Neural Architecture Search from Message-passing” adds that optimal depth is dataset-dependent and that deeper propagation can worsen performance because of over-smoothing, so manual enumeration of depth is inherently restrictive [2103.14282].

Theoretical analyses in later adaptive-depth work sharpen this point. Curvature-based depth adaptation establishes a bound linking local mixing time to curvature,
\[
\tau_x(\epsilon) \le \frac{\log(1/\epsilon)}{\kappa(x)},
\]
which is used to argue that high-curvature vertices mix rapidly and therefore require fewer layers, while low-curvature vertices benefit from deeper propagation [2503.01079]. A separate homophily-sensitive theory defines a local signal preservation factor
\[
\alpha_v=\frac{1+d_v^+-d_v^-}{d_v+1},
\]
and shows that the usefulness of aggregation depends jointly on same-label neighbors, opposite-label neighbors, and degree, so a single depth can over-aggregate some nodes and under-aggregate others [2511.06608].

## 3. Core architectural patterns

The direct ADMP-GNN formulation in [2509.01170] is a model-agnostic wrapper over standard message-passing GNNs. The underlying message passing follows
\[
m^{(\ell)}_v = \psi^{(\ell)}(\{h_u^{(\ell-1)} : u \in \mathcal{N}(v)\}), \qquad h^{(\ell)}_v = \phi^{(\ell)}(h_v^{(\ell-1)}, m_v^{(\ell)}),
\]
with \(h_v^{(0)}=x_v\) [2509.01170]. The distinctive addition is a prediction head at every layer:
\[
p^{(\ell)}_v = \phi^{(\ell)}_{\mathbf{Ex}(h_v^{(\ell-1)}, m_v^{(\ell)}) = \text{Softmax}(\widetilde{W}^{(\ell)} m_v^{(\ell)}),
\]
together with a continuation update
\[
h_v^{(\ell+1)} = \phi^{(\ell)}_{\mathbf{Ct}(h_v^{(\ell-1)}, m_v^{(\ell)}),
\]
and a layer-0 classifier
\[
p_v^{(0)} = \text{Softmax}(\widetilde{W}^{(0)} x_v)
\]
[2509.01170]. This yields a sequence of layer-wise classifiers from depth \(0\) through \(L\), after which each node is assigned an exit layer.

Other ADMP-GNN variants implement adaptivity differently. In curvature-guided adaptive depth, the architecture remains a standard GNN backbone such as GCN, GAT, GraphSAGE, GIN, SGC, MixHop, or DIR-GNN, but a curvature estimation module computes \(\hat{\kappa}(x)\) per vertex, converts it to a stopping depth \(T(x)\), and runs adaptive message passing in which neighbors contribute either their current-layer representation or their final stopped representation [2503.01079]. The update is
\[
m_x^{(t)} = \text{AGG}\Bigl( \{\!\{h_y^{(\min\{t-1,\,T(y)\})} \mid y\in N(x)\}\!\} \Bigr), \qquad
h_x^{(t)} = \text{UPD}\Bigl(h_x^{(t)},\, m_x^{(t-1)}\Bigr)
\]
for \(t\le T(x)\), and prediction uses \(\hat{y}_x = \mathcal{C}(h_x^{(T(x))})\) [2503.01079].

In the variational AMP framework, adaptive depth is coupled with soft message filtering. The filtered update is
\[
\mathbf{h}_v^{\ell} = \phi^{\ell}\!\left( \mathbf{h}_v^{\ell-1},\, \Psi\!\left( \left\{ \mathbf F_i(u,\ell-1)\odot \psi^{\ell}(\mathbf{h}_u^{\ell-1}, \mathbf a_{uv}^{\ell}) \mid u \in \mathcal N_v \right\} \right) \right),
\]
where \(\mathbf F_i(u,\ell-1)\in[0,1]^d\) modulates outgoing messages from node \(u\) at layer \(\ell-1\) [2312.16560]. This makes adaptation act simultaneously on depth and message content.

In differentiable architecture search, the Graph Neural Architecture Paradigm (GAP) defines a directed tree with feature filtering before aggregation, one neighbor-aggregation stage per branch, and feature filtering afterward [2103.14282]. In this setting, message-passing depth is “the number of stacked graph architectures that contain neighbor aggregation,” so depth is determined by the selected operations rather than imposed externally [2103.14282].

## 4. Depth selection mechanisms

ADMP-GNN methods differ most sharply in how they decide the stopping depth.

In the multi-exit ADMP-GNN of [2509.01170], depth selection is not learned through an internal halting controller at inference time. Instead, the model assumes that structurally similar nodes should share the same best exit layer. Structural similarity is approximated using centrality metrics—degree, \(k\)-core, PageRank, and Walk Count for length 2—after which nodes are sorted by centrality, divided into \(C\) equal-sized clusters, and validation nodes are used to select the best layer for each cluster:
\[
\ell_c = \argmax_{\ell\in\{0,\ldots,L\} } \text{Acc}\left\{ p_v^{(\ell)} \;:\; v \in \mathcal{V}_{val} \cap \mathcal{P}_c \right\}.
\]
Test nodes in cluster \(c\) then use exit layer \(\ell_c\) [2509.01170]. The paper argues that validation nodes are preferable to training nodes for this policy because training accuracy is overly optimistic across layers [2509.01170].

The curvature-based method of [2503.01079] uses a rank-based depth assignment rule:
\[
T(x) \coloneqq \min\left\{ t \in \mathbb{N} \;\middle|\; \frac{1}{|V|}\sum_{y \in V}\mathbb{I}(\kappa(y)\ge \kappa(x)) \le \frac{k\,t}{100} \right\}.
\]
Here \(k\) is a threshold hyperparameter. Smaller \(k\) tends to allow deeper propagation for more vertices, whereas larger \(k\) yields earlier stopping [2503.01079]. The theoretical basis is a curvature-dependent exponential decay of feature distinctiveness,
\[
D(x,l) \le e^{-\kappa(x) l \Delta t},
\]
which motivates smaller \(T(x)\) for larger \(\kappa(x)\) [2503.01079].

The variational AMP framework selects depth by learning a posterior \(q(L;\lambda)\) over possible depths rather than committing to a single integer depth [2312.16560]. The support of \(q(L)\) is truncated at a high quantile such as \(0.99\), and prediction averages outputs across all supported depths with weights determined by the learned posterior:
\[
p(Y_j\mid g_j,\mathcal D) \approx \sum_{\ell=1}^{\hat L} q(\ell;\boldsymbol\lambda)\, p\!\left(Y_j;\Omega_\ell(g_j,\mathbf F_j,\boldsymbol\nu)\right)
\]
[2312.16560]. Depth adaptation here is distributional rather than node-wise.

The message-passing NAS perspective in [2103.14282] yields another mechanism: initialize with a depth related to graph diameter, search a super-network, derive a discrete subarchitecture, count the number of cells that actually contain neighbor aggregation, and use that count as the next depth until convergence. In the reported algorithm, \(\mathcal{D}_o\) is initialized as half of the average graph diameter and then updated to the number of cells with at least one neighbor aggregation in the derived network [2103.14282]. This makes depth an emergent property of the architecture search process.

## 5. Training formulations and optimization

The direct ADMP-GNN paper introduces two training paradigms for multi-exit models [2509.01170]. Aggregate Loss Minimization (ALM) jointly minimizes the sum of losses across all exits:
\[
\mathcal{L}_{\text{ALM} := \argmin_{\theta} \mathbb{E}_{v\in\mathcal{V} \left[ \sum_{\ell=0}^{L} \mathcal{L}\left( p_v^{(\ell)}(m_v^{(0)}, \theta_0,\ldots,\theta_\ell), y_v \right) \right],
\]
with \(\mathcal{L}\) equal to cross-entropy [2509.01170]. The paper reports that this simple joint objective suffers from gradient conflicts, especially because early layers are influenced by many deeper losses [2509.01170].

Sequential Training (ST) is the preferred alternative. It trains layer \(0\), freezes it, then trains each deeper layer after previous layers are fixed, using the recursion
\[
\forall v\in\mathcal{V},\quad S_{\ell+1}(v) = \mathcal{L}\left( p_v^{(\ell)}(m_v^{(0)}, \theta_0^\star,\ldots,\theta_\ell^\star,\theta_{\ell+1}), y_v \right) + S_\ell(v),
\]
with
\[
\theta_\ell^\star = \argmin_{\theta_\ell} \mathbb{E}_{v\in\mathcal{V}[S_\ell(v)],
\qquad
S_0(v)=\mathcal{L}(\phi^{(0)}_{\mathbf{Ex}, y_v)
\]
[2509.01170]. The reported trade-off is that ST involves \(L+1\) training phases and thus higher wall-clock time, but yields better multi-exit behavior and greater stability [2509.01170].

Curvature-guided adaptive depth augments the task loss with curvature regularization:
\[
\mathcal{L}_{\text{total} = \mathcal{L}_{\text{task} + \mathcal{L}_{\text{curv},
\]
where
\[
\mathcal{L}_{\text{curv} = \sum_{x\in V} \left( \sum_{f\in\mathcal{F} \mathcal{L}(\hat{\kappa}(x),f) - \lambda\,\hat{\kappa}(x) \right)
\]
and \(\lambda=1\) in experiments [2503.01079]. The finite family \(\mathcal F\) of MLP-generated functions approximates the otherwise intractable curvature optimization, and \(N\) is small in experiments, for example \(N\le 5\) [2503.01079].

The AMP framework uses variational inference and optimizes an ELBO over latent parameters \(\Theta\), depth \(L\), and message filters \(\mathbf F_i\) [2312.16560]. After factorization, the practical objective becomes a finite sum over supported depths,
\[
\sum_{\ell=1}^{\hat L} q(\ell;\boldsymbol\lambda) \left[ \ln\frac{p(\ell)}{q(\ell;\boldsymbol\lambda)} + \ln\frac{p(\boldsymbol\nu)}{q(\boldsymbol\nu\mid \ell;\boldsymbol\nu)} + \ln\frac{p(\mathbf F_i)}{q(\mathbf F_i\mid g_i,\ell,\boldsymbol\nu)} + \ln p(Y_i\mid \ell,\mathbf F_i,\boldsymbol\nu,g_i) \right]
\]
[2312.16560]. This formalism treats all depths in the supported range as active contributors during training.

The differentiable NAS formulation in [2103.14282] adopts a DARTS-like relaxation,
\[
\bar{o}^{(i,j)}(x) = \sum_{o \in \mathcal{O}softmax(\alpha_{o}^{(i,j)})o(x),
\]
jointly optimizing network weights \(w\) and architecture parameters \(\alpha\), then recovering a discrete architecture by
\[
o^{(i,j)} = argmax_{o \in \mathcal{O} (\alpha_{o}^{(i,j)}).
\]
Only the strongest non-zero operation is retained per edge, and only the strongest incoming edge per node is kept [2103.14282].

## 6. Empirical findings and comparative picture

The direct ADMP-GNN evaluation focuses on node classification over thirteen benchmarks including Cora, CiteSeer, PubMed, CS, ogbn-arxiv, Genius, Amazon Computers, Amazon Photo, Chameleon, Squirrel, Cornell, Texas, and Wisconsin, with the main reported tables emphasizing six datasets: Cora, CiteSeer, CS, PubMed, Genius, and ogbn-arxiv [2509.01170]. Baselines include GCN, GIN, JKNet variants, GCNII / Residuals, AdaGCN, and GPR-GNN, and baseline depth is grid-searched from 0 to 5 for fairness [2509.01170]. The main evaluation metric is classification accuracy as mean ± standard deviation over 10 runs, using Adam and an NVIDIA A100, with maximum depth fixed to \(L=5\) for ADMP-GNN in the principal comparisons [2509.01170].

For the GCN backbone, the paper reports the following accuracies for ADMP-GNN versus GCN: Cora 81.22 vs 80.78, CiteSeer 71.33 vs 71.25, CS 91.49 vs 92.20, PubMed 79.02 vs 79.32, Genius 80.17 vs 80.76, and ogbn-arxiv 69.29 vs 64.37 [2509.01170]. For the GIN backbone, ADMP-GIN is reported as 78.07 vs 77.73 on Cora, 65.41 vs 65.23 on CiteSeer, 90.82 vs 90.29 on CS, 76.46 vs 76.05 on PubMed, and 60.85 vs 60.70 on ogbn-arxiv [2509.01170]. The same paper states that Sequential Training consistently outperforms Aggregate Loss Minimization and that oracle accuracies are much higher than any fixed-depth model, with GCN oracle values of 89.43 on Cora, 81.96 on CiteSeer, 97.24 on CS, 90.13 on PubMed, 85.97 on Genius, and 79.64 on ogbn-arxiv [2509.01170].

The curvature-guided method reports evaluation on 21 datasets spanning vertex classification, vertex regression, graph classification, and graph regression, and states consistent gains over GCN, GAT, GraphSAGE, SGC, MixHop, and DIR-GNN on homophilic graphs, heterophilic graphs, and large-scale ogbn-arxiv [2503.01079]. It further reports that improvements are often larger on heterophilic graphs, that oversmoothing robustness persists up to 64 layers, and that replacing learnable Bakry–Émery curvature with Ollivier-Ricci, Forman-Ricci, Jost–Liu curvature, effective resistance, betweenness centrality, or degree performs worse overall [2503.01079].

The variational AMP framework evaluates five long-range datasets—Diameter, SSSP, Eccentricity, peptides-func, and peptides-struct—and reports average test-error reductions of 63% on Diameter, 72% on SSSP, and 32% on Eccentricity relative to base architectures [2312.16560]. On chemical tasks, it reports improvements of about 2–3.4% AP on peptides-func and state-of-the-art-level MAE on peptides-struct [2312.16560]. These results address a broader long-range setting than node classification alone.

In message-passing NAS, GNAS is evaluated on ZINC, PATTERN, CLUSTER, MNIST, and CIFAR10 and is reported to outperform manual GNNs such as GCN, GIN, GraphSAGE, GAT, GatedGCN, and MoNet, as well as the search-based baseline GraphNAS, while using much less search time [2103.14282]. On ZINC, the searched depth reportedly converges to around 12–14 regardless of the initial depth when the initialization is not too small, supporting the claim that effective depth can be learned rather than preset [2103.14282].

## 7. Interpretations, limitations, and relation to adjacent work

A central misconception is that adaptive depth necessarily means a fully learned per-node halting policy inside the network. The direct ADMP-GNN formulation explicitly does not use “a learned per-node halting gate inside the network at inference time”; instead, it combines multi-layer exits, a training scheme to make each layer a strong predictor, and a heuristic policy based on structural similarity [2509.01170]. By contrast, curvature-guided stopping is more intrinsically node-wise, while variational AMP learns a posterior over depths rather than a hard node-specific halting layer [2503.01079; 2312.16560]. The term therefore covers multiple adaptation mechanisms rather than one canonical controller.

Another misconception is that adaptive depth and message filtering are interchangeable. The literature suggests they are related but distinct. In [2312.16560], depth adaptation addresses underreaching by learning how many layers are needed, while message filtering suppresses irrelevant or harmful information to mitigate oversmoothing and oversquashing. In [2103.14282], the analogous decomposition is between feature filtering and neighbor aggregation, with the former controlling what is sent or preserved and the latter determining the actual message-passing step. This suggests that adaptive-depth message passing is often most effective when depth control is coupled with some form of content control.

The main limitations are method-specific. The direct ADMP-GNN paper assumes that structural similarity correlates with optimal depth, relies on validation-node-driven cluster policies, and incurs heavier training cost under Sequential Training; it is also evaluated mainly on node classification [2509.01170]. The curvature-based approach approximates curvature using a finite family of MLP-generated functions and therefore computes an upper bound on the true curvature rather than an exact quantity [2503.01079]. The AMP framework introduces additional variational machinery and probabilistic assumptions, including bounded-support depth distributions and deterministic Dirac message-filter posteriors [2312.16560]. The homophily-sensitive theory in [2511.06608] is derived under a simplified CSBM with binary labels and row-normalized averaging, omitting symmetric normalization, learned weights, and nonlinearities.

Taken together, these works position ADMP-GNN as a broad research direction at the intersection of dynamic computation, long-range graph representation learning, and message-passing control. The unifying idea is that depth should not be treated as a universal constant. Instead, it can be assigned by structural heuristics [2509.01170], geometric quantities such as Bakry–Émery curvature [2503.01079], latent-variable inference over depth distributions and filters [2312.16560], or architecture search over message-passing primitives [2103.14282]. A plausible implication is that future ADMP-GNN research will continue to move from manually chosen propagation horizons toward data-dependent, theoretically informed depth policies that operate at finer granularity than the graph-wide hyperparameter choices of conventional GNNs.

Source: https://www.emergentmind.com/topics/adaptive-depth-message-passing-gnn-admp-gnn