Delta Graph Network (DeltaGN) Overview

• Delta Graph Network (DeltaGN) is a dual-branch graph neural network that effectively mitigates over-smoothing and over-squashing using an Information Flow Control module.
• It employs an Information Flow Score and a condensation procedure to dynamically filter edges and isolate critical nodes for preserving long-range interactions.
• DeltaGN maintains O(|V|+|E|) per-layer complexity and shows superior performance on diverse node classification benchmarks, highlighting its scalability and robustness.

Delta Graph Network (DeltaGN), introduced as DeltaGNN, is a dual-branch Graph Neural Network (GNN) incorporating an Information Flow Control (IFC) module designed to effectively address over-smoothing and over-squashing in message passing on graphs of arbitrary size and topology. DeltaGNN achieves scalable and generalizable detection of both long-range and short-range node interactions by leveraging an embedding-based Information Flow Score (IFS) for dynamic edge filtering and a condensation procedure to isolate critical nodes for preserving long-range dependencies. The architecture attains $O(|\mathcal V|+|\mathcal E|)$ per-layer complexity, matching standard GNN models, and exhibits consistent empirical superiority across diverse node classification benchmarks (Mancini et al., 10 Jan 2025).

1. Architectural Structure and Pipeline

DeltaGNN comprises a dual-branch structure integrating information gating and graph condensation mechanisms:

• Homophilic Aggregation Branch with IFC: At each message-passing layer, a standard GNN transformation (e.g., GCN or GIN) is interleaved with an IFC module. IFC computes the IFS for each node and filters edges prone to inducing over-smoothing (predominantly heterophilic edges) and over-squashing (adjacent to bottlenecks).
• Heterophilic Graph Condensation and Aggregation Branch: A graph condensation operation selects a subset of nodes with high IFS to compose a fully-connected, "condensed" graph, explicitly preserving long-range interactions. A second aggregation branch then operates over this condensed structure to encode long-range dependencies.
• Final Readout: Features from both branches are concatenated for downstream node classification.

Formally, for each layer $t$ ($t=1,\dots,T$), the forward sequence is

$$(\mathbf A^{t-1},\mathbf X^{t-1}) \xrightarrow{\Omega_t} (\mathbf A^{t-1},\mathbf X^t) \xrightarrow{\Theta_t} (\mathbf A^t,\mathbf X^t).$$

2. Information Flow Control and Information Flow Score

DeltaGNN’s IFC is an embedding-aware edge gating mechanism constructed around the Information Flow Score:

• First-Delta ("velocity"): For each node $u$, the first delta at layer $t$ is

$$\Delta^t_u = d\bigl(\bigoplus_{v\in\mathcal N(u)}\mathbf M^t_v,\;\mathbf M^t_u\bigr),$$

where $\mathbf M^t_u=\psi(\mathbf X_u^{t-1})$ and $d$ is a distance function.

• Second-Delta ("acceleration"): Defined recursively as

$$(\Delta^2)^t_u = \begin{cases} 0, & t=1\ d(\Delta^t_u, \Delta^{t-1}_u), & t\ge 2 \end{cases}$$

• Information Flow Score (IFS): For node $u$,

$$S_u = \frac{m\,\mathbb V_t[(\Delta^2)^t_u] + 1} {l\,\overline{\Delta_u} + 1},$$

where $$\overline{\Delta_u} = \frac{1}{T} \sum_t \Delta^t_u$$ is the mean first-delta, $\mathbb V_t[(\Delta^2)^t_u]$ its variance over layers, and $l, m > 0$ are balancing constants (with $l = m = 1$ in practice).

A low $S_u$ identifies either pronounced over-squashing (low variance in acceleration near bottlenecks) or over-smoothing (high mean "velocity" in heterophilic neighborhoods).

3. Edge Filtering and Training Dynamics

At each GNN layer after the computation of IFS values, IFC removes $K(t,\theta)$ edges with the lowest incident node scores. The edge-budgeting function $K(t,\theta)$ can be constant or ramp linearly with depth, parameterized by $\theta$—which is updated via local hill-ascent on a utility objective, such as mean terminal node score. The typical update within layer $t$ is:

1. Transform and aggregate node features,
2. Compute first and second deltas,
3. Update mean/variance using Welford’s method,
4. Calculate IFS per node,
5. Remove $K(t,\theta)$ edges with lowest node scores to yield the updated adjacency.

This strategy jointly targets bottlenecked and overly mixed regions, adaptively regulating information propagation and preserving salient structural information.

4. Theoretical Foundations

DeltaGNN establishes the information-flow approach through the following lemmas:

• Lemma 1 (Over-smoothing and Homophily): For a class-separating classifier $\Phi$, if $\overline{\Delta_u} > \rho$ for some threshold $\rho > 0$, the local homophily $\mathcal H_u < \mathcal H$ (i.e., high velocity implies predominantly heterophilic context).
• Lemma 2 (Over-squashing and Connectivity): For a node $u$ with connectivity $c(u) < \mu$ (bottleneck threshold), $\mathbb V_t[(\Delta^2)^t_u]$ is small, highlighting over-squashed locations.

Therefore, the ratio in the IFS definition is theoretically motivated to flag problematic graph regions for targeted filtration.

5. Algorithmic Complexity

The computational and memory cost per GNN layer is summarized as:

Operation	Time Complexity	Notes
Delta computations	$O(|\mathcal V|\,d)$	$d\ll|\mathcal V|$
Welford’s mean/var update	$O(|\mathcal V|)$	Two scalars per node
Edge filtering	$O(|\mathcal E|)$	$O(|\mathcal E|\log|\mathcal V|)$ if sorted
Overall per layer	$O(|\mathcal V|+|\mathcal E|)$	Matching standard GNN

Only $O(|\mathcal V|)$ additional storage is required for tracking mean and variance per node. The design is intended to ensure practical runtime on large and dense graphs.

6. Empirical Evaluation

DeltaGNN was evaluated on 10 node-classification benchmarks, including citation networks (Cora, CiteSeer, PubMed), webpage graphs (Cornell, Texas, Wisconsin), and image-based graphs (MedMNIST: Organ-S, Organ-C) with broad homophily ranges and densities.

Key empirical results:

• On six homophily-varying datasets, DeltaGNN with a linear $K(t)$ policy achieved best test accuracy on four, with average improvement of +1.23% over diverse GNN and transformer baselines.
• On large/dense MedMNIST graphs, DeltaGNN matched or outperformed other LRI-capable models, being the only method not to trigger out-of-memory or timeouts, and improved average accuracy by +0.92%.
• Ablations revealed linear-ramp $K(t,\theta)$ yields superior trade-offs.
• Substituting IFS with standard graph-theoretic scores (degree, eigenvector, betweenness, closeness, Forman-Ricci, Ollivier-Ricci) led to substantially lower and inconsistent performance.
• An experiment on a 14-node toy graph showed IFS-based edge filtering increases graph-level homophily by +8.65% (versus +4–6% for alternative scores) and reduces bottlenecks.

7. Limitations and Prospects for Extension

Identified limitations:

• Although per-layer complexity in $|\mathcal V|$ is linear, filtering $|\mathcal E|$ edges at every layer can be non-negligible for extremely large graphs.
• Local hill-ascent optimization of $\theta$ may be susceptible to local maxima, indicating need for more global hyperparameter search.

Potential extensions include applying IFC to non-node-level tasks (e.g., graph classification, link prediction), integrating alternative aggregation schemas such as attention or spectral methods within IFC, continuous relaxation of the edge-filtering budget, and further theoretical analysis of the convergence and expressive capacity under iterative filtration (Mancini et al., 10 Jan 2025).