Over-Smoothing Decay Rate in GNNs

• Over-smoothing decay rate is a measure quantifying the exponential convergence of GNN node features to a homogeneous limit, driven by spectral parameters and graph topology.
• It is characterized by the decay of feature differences according to sub-dominant eigenvalues and metrics like Dirichlet energy, revealing the effects of graph structure and operator norms.
• Control strategies like residual connections, edge dropping, and nonlocal propagation techniques are effective in mitigating over-smoothing and preserving discriminative power.

Over-smoothing decay rate quantifies the exponential rate at which node feature differences or higher-order nontrivial directions collapse toward indistinguishability in deep graph neural networks (GNNs). In both theoretical analysis and empirical investigation, this rate emerges as a precise function of spectral or structural parameters of the propagation operator, feature transformations, and graph topology. Over-smoothing is thus directly characterized by how embeddings converge toward a low-rank or homogeneous limit as the number of layers increases.

1. Formal Definitions and Operator-Theoretic Foundations

Over-smoothing in GNNs refers to the phenomenon where, as layers are stacked, the representations $H^{(k)}$ of nodes become increasingly similar, often collapsing to a subspace with limited discriminative power. The canonical formalization is as follows:

• Graph convolutional update:

$H^{(k+1)} = \widehat{A} H^{(k)} W^{(k)}$

where $\widehat{A}$ is the normalized adjacency or aggregation matrix and $W^{(k)}$ a learnable feature transform.

• Power iteration perspective:

When $W^{(k)}=W$ (or $W$ orthogonal), $H^{(K)} = \widehat{A}^K H^{(0)} W^K$. Vectorizing yields $\mathrm{vec}(H^{(K)}) = S^K \mathrm{vec}(H^{(0)})$ with $S = W^\top \otimes \widehat{A}$, and the model implements power iteration by $S$ (Roth, 2024).

• Exponential decay rate:

The convergence rate of $H^{(k)}$ toward the limit is governed by the sub-dominant eigenvalues: $$\|H^{(k)}/\|H^{(k)}\| - H^*\| = O\left((|\lambda_2(S)|/|\lambda_1(S)|)^k\right)$$, where $H^*$ is the asymptotic rank-one limit.

• Rank collapse:

More generally, a sequence $X^{(k)}$ "rank-collapses" if $\| X^{(k)}/\|X^{(k)}\| - Y^{(k)} \| \to 0$ for rank-one $Y^{(k)}$, capturing both over-smoothing and over-correlation (Roth, 2024, Roth et al., 2023).

2. Spectral Parameters and Universal Exponential Laws

The decay rate depends on the spectral properties of the propagation operator. Let $S$, $\widehat{A}$, or $P$ denote the relevant matrix (normalized adjacency, Laplacian, or random walk operator):

• Graph Laplacian and Dirichlet energy decay:

For continuous-time diffusion $\frac{\partial f_t}{\partial t} = \Delta_\mu f_t$, the Dirichlet energy satisfies $E_1(f_t) \le e^{-2\lambda_1 t} E_1(f_0)$, where $\lambda_1$ is the first non-trivial eigenvalue of $-\Delta_\mu$ (Guan et al., 7 Dec 2025, Shao et al., 2023).

• Discrete propagation:

For $P_\mu = \Delta_\mu + I$, $$\int_G \|\nabla_\mu P^k f\|^2 d\mu \le (1 - (2 - \lambda_N)\lambda_1)^k \int_G \|\nabla_\mu f\|^2 d\mu$$ with $\lambda_N$ the largest Laplacian eigenvalue; the decay factor is always strictly less than $1$ on non-bipartite graphs (Guan et al., 7 Dec 2025).

• Normalized adjacency and general GNN:

For linear propagation ($H^{(k+1)} = S H^{(k)}$), if $s$ is the supremum of layer operator norms and $\lambda$ is the second-largest eigenvalue modulus of $S$, $d_{\mathcal{M}}(H^{(k)}) \leq C \rho^k$, with $\rho = s \lambda$ and $d_{\mathcal{M}}$ quantifying distance from the smoothed subspace (Sun, 2022, Huang et al., 2020).

• Feature difference decay:

For regular graphs or strong curvature (see below), $\|x_u^{(k)} - x_v^{(k)}\| \leq C\, r^k$, with $r$ depending on overlap (curvature-based bounds) or on $|\lambda_2|$ (Nguyen et al., 2022).

3. Metrics: Dirichlet Energy, High-Frequency Content, and Rank-One Distance

Several metrics precisely characterize the collapse rate:

Metric Name	Mathematical Definition	Collapse Rate
Dirichlet Energy ($E_1$)	$$E_1(H) = \sum_{(i,j)\in E} \omega_{ij}\|h_i - h_j\|^2$$	$O(\rho^{2k})$, $\rho = |\lambda_2|$
High-Frequency Energy	$E_{\mathrm{high}}(F^{(k)}) = \|P F^{(k)}\|_F^2$, $P$ projects onto high eigenspaces	$O(|\mu_2|^{2k})$
Rank-One Distance (ROD)	$ROD(X) = \| X/\|X\| - (uv^\top)/(\|uv^\top\|)\|$ for max-norm (row $u$, column $v$)	$O(\rho^k)$, $\rho \approx |\lambda_2|$

• Empirically, all these metrics show clean exponential decay on standard GNNs, with the observed exponent matching the theoretical spectral rate (Roth, 2024, Roth et al., 2023, Yang et al., 2024).
• Residual connections, normalization schemes, or explicit anti-collapse measures can alter or arrest this decay, preventing over-smoothing (Roth, 2024, Huang et al., 2020).

4. Structural and Geometric Factors Affecting Decay

The spectral gap, graph topology, normalization, and even graph curvature control over-smoothing rates:

• Spectral gap ($\gamma=1-\lambda$):

Larger gap accelerates smoothing (smaller $|\lambda_2|$), as seen in complete or small-world graphs; sparse or large-diameter graphs slow it (Sun, 2022, Huang et al., 2020).

• Ollivier-Ricci curvature:

On regular graphs with lower-bounded positive curvature $\delta$, feature differences between neighbors contract geometrically as $r(\delta)^k$ with $$r(\delta) = \frac{3LM\lfloor (1-\delta)n\rfloor}{n+1} < 1$$: more positive curvature, faster collapse (Nguyen et al., 2022).

• Nonlocal/Algebraic Smoothing:

Proposed nonlocal PDE dynamics, with adaptive energy-dependent propagation, yield algebraic decay $O(1/L)$ rather than exponential, thus fundamentally attenuating over-smoothing (Guan et al., 9 Dec 2025).

• Operator Consistency:

Operator-consistent GNNs (fixed $P$) always exhibit exponential over-smoothing; operator-inconsistent (e.g., layerwise-varying attention) can avoid it if per-layer stationary distributions do not converge (Zhao et al., 2022).

5. Theoretical and Empirical Validation Across GNN Variants

Empirical studies and theoretical analysis consistently validate the spectral decay law:

• Experiments on Cora, Citeseer, Pubmed, and synthetic benchmarks confirm that Dirichlet energy or high-frequency energy decays linearly on a log-scale, with slope precisely predicted by $|\lambda_2|$ or related spectral quantities (Guan et al., 7 Dec 2025, Yang et al., 2024, Roth et al., 2023).
• For plain GCN, GAT, SAGE, and similar models, the collapse rate matches $O(\rho^k)$ with $\rho \approx |\lambda_2|$ (Roth, 2024, Roth et al., 2023).
• Skip-connections, adaptive aggregation, and DropEdge slow the decay (increasing $\rho$), while true algebraic smoothing (nonlocal models) further reduce the rate to $O(1/L)$ (Guan et al., 9 Dec 2025, Huang et al., 2020).
• In practice, over-smoothing in typical GCN architectures emerges within a few (4–8) layers for graphs with moderate spectral gap (Arroyo et al., 15 Feb 2025).

6. Implications, Control Strategies, and Generalizations

The explicit connection between over-smoothing decay rate and graph/model parameters enables effective interventions and broader generalization:

• Slowing over-smoothing:

Decrease spectral gap (e.g., edge dropping/Sparsification), introduce residual or skip connections, design multi-aggregator (sum of Kronecker products) architectures, and employ nonlocal propagation (Roth, 2024, Roth et al., 2023, Guan et al., 9 Dec 2025).

• Avoidance regimes:

Time-inhomogeneous processes or GATs with layerwise-regularized attention can avoid exponential collapse if a uniform gap between per-layer stationary distributions is enforced (Zhao et al., 2022).

• Unified view with vanishing gradients:

The decay rate of over-smoothing is fundamentally linked with gradient vanishing phenomena; when the contraction constant $L<1$ per layer, both feature and gradient norms decay exponentially (Arroyo et al., 15 Feb 2025).

• Hilbert and Banach space generalization:

In dynamical/functional analysis, smoothing by exponentially-stable semigroups yields decay of the form $O(t^{-a/2})$ for regularizing operators, reinforcing the universality of the smoothing rate (Wakaiki, 2022).

---

In summary, the over-smoothing decay rate is a sharp, model-agnostic descriptor of the exponential contraction of GNN feature differences and higher-order graph functionals, directly controlled by the spectral (or geometric) characteristics of the graph and the structure of the propagation operator. This rate interfaces with fundamental limitations in learning, expressivity, and gradient flow in deep GNNs, and drives the design of contemporary anti-over-smoothing techniques across both theory and practice (Roth, 2024, Guan et al., 7 Dec 2025, Sun, 2022, Roth et al., 2023, Guan et al., 9 Dec 2025, Zhao et al., 2022, Yang et al., 2024, Arroyo et al., 15 Feb 2025, Nguyen et al., 2022, Huang et al., 2020, Wakaiki, 2022).