Graph Cohesive Smoothing in GNNs

• Graph Cohesive Smoothing (GCS) is a framework that regulates node representation smoothness in GNNs by balancing noise suppression and signal retention through controlled aggregation and spectral design.
• It leverages finite smoothing to denoise data by compressing minor eigenvalue modes and uses metrics like MAD and MADGap to quantitatively assess and guide smoothing dynamics.
• GCS employs techniques such as MADReg, AdaEdge, and learnable smoothness control (SCT) to adaptively maintain class-cohesiveness while mitigating oversmoothing in deep graph networks.

Graph Cohesive Smoothing (GCS) refers to a principled framework for regulating the smoothness of node representations produced by graph neural networks (GNNs), governed by the interplay between aggregation, spectral properties of the underlying graph, the applied transformations (such as mean aggregation and nonlinearity), and learning objectives that enforce class-cohesiveness. The central goal is to amplify beneficial structure while suppressing noise, but without oversmoothing, which erases discriminative information by collapsing node representations. GCS spans theoretical analysis (linear mean-aggregation models), quantitative metrics, algorithmic modifications to message passing, and practical strategies for monitoring and adapting the degree of smoothing for improved regression and classification on graphs (Keriven, 2022, Chen et al., 2019, Wang et al., 2024).

1. Linear Graph Smoothing and the Oversmoothing Limit

The foundational model for GCS is repeated mean aggregation on an undirected graph, with node features $Z \in \mathbb{R}^{n \times p}$. Given adjacency $A$ and degree $D=\mathrm{diag}(d)$, the random-walk operator $L = D^{-1}A$ is applied iteratively: $Z^{(k)} = L^k Z$. If $x\in\mathbb{R}^n$ is a scalar feature, then $x^{(k)}=L^k x$.

This process admits a spectral decomposition when the graph is connected. $L$ has eigenvalue $1$ (with eigenvector $1_n$), all others $A$0. Thus, as $A$1,

$A$2

and all node features collapse to a limiting constant—this is the phenomenon of oversmoothing (Keriven, 2022).

In ridge-regression or classification setups, this collapse is formalized: predicted labels after many iterations converge to constants and the testing risk approaches the variance of the labels, obliterating useful signal.

2. Finite Smoothing and Spectral Denosing

GCS demonstrates that for finite $A$3, repeated smoothing offers denoising benefits via two spectral mechanisms:

• Compression of Minor Modes (Regression): For data drawn from $A$4, the covariance post-smoothing becomes

$A$5

so the smallest eigenvalues shrink fastest. Thus, high-frequency (noise) components fade rapidly, while principal directions (signal) are preserved longer. Optimal regression risk is achieved when noise eigendirections shrink to the level of the ridge regularization $A$6 (Keriven, 2022).

• Cohesive Within-Community Collapse (Classification): In latent space block models with within-class covariance $A$7, mean-aggregation reduces this to $A$8 per layer. Inter-class mean separation stays fixed, so initial risk decreases as within-class dispersion shrinks; only after many layers does global collapse negate class separation. The optimal number of smoothing layers is thus data- and regularization-dependent.

This spectral view yields explicit formulas for risk as a function of smoothing depth $A$9, eigenvalues, and $D=\mathrm{diag}(d)$0. The optimal $D=\mathrm{diag}(d)$1 balances noise suppression and information retention and can be approximated analytically.

3. Metrics for Cohesive Smoothness: MAD and MADGap

To formally quantify and guide cohesive smoothing, (Chen et al., 2019) introduces two key metrics, calculated on node embeddings $D=\mathrm{diag}(d)$2:

• Mean Average Distance (MAD): For a binary mask $D=\mathrm{diag}(d)$3 (specifying pairs of interest, e.g. all-pairs or intra-class), MAD is the average cosine distance between feature vectors, measuring global or local smoothness.
• MADGap: Defined as the difference between the MAD of remote node pairs (different communities) and that of close pairs (likely same class),

$D=\mathrm{diag}(d)$4

A large MADGap indicates strong class separation; its reduction signals oversmoothing. GCS thus aims to raise or maintain large MADGap values during learning.

These metrics guide and evaluate both architectural interventions and training dynamics.

4. Methods for Steering Graph Cohesive Smoothing

To enforce cohesive but not excessive smoothing, GCS encompasses several algorithmic techniques:

• Regularization-based Control (MADReg): Adds a negative MADGap term to the loss,

$D=\mathrm{diag}(d)$5

This penalizes representations that collapse remote pairs while encouraging intra-class smoothness. MADReg is architecture-agnostic and gradient computation leverages the form of cosine distances (Chen et al., 2019).

• Adaptive Graph Editing (AdaEdge): Dynamically refines the graph adjacency according to model predictions, preferentially adding intra-class and removing inter-class edges to better support cohesive smoothing.
• Spectral Polynomial Filters: Instead of simple powers $D=\mathrm{diag}(d)$6, polynomial filters $D=\mathrm{diag}(d)$7 selective attenuate minor modes (high-frequency noise) while preserving main spectral components. This enables finer spectral control as compared to fixed-depth smoothing (Keriven, 2022).
• Learnable Smoothness Control Term (SCT): In GCN architectures, an explicit bias $D=\mathrm{diag}(d)$8 constrained to the smooth eigenspace $D=\mathrm{diag}(d)$9 (spanned by the eigenvectors with eigenvalue 1 of the normalized adjacency $L = D^{-1}A$0) is added at each layer:

$L = D^{-1}A$1

Parameterized by pooling the previous layer into $L = D^{-1}A$2, an MLP outputs the learnable bias, enabling the model to tune the ratio of smooth to non-smooth features dynamically per task and per layer (Wang et al., 2024).

5. Geometric and Spectral Analysis of GCS in Deep GNNs

A detailed geometric analysis of ReLU and leaky-ReLU activations underpins the SCT mechanism. Following (Wang et al., 2024), consider feature vectors decomposed as $L = D^{-1}A$3. ReLU and leaky-ReLU reduce the $L = D^{-1}A$4 norm but, via an appropriate shift in the smooth component, allow the normalized smoothness $L = D^{-1}A$5 to span any task-required interval.

Formally, for ReLU, $L = D^{-1}A$6 can range continuously between a value determined by the entrywise maximum and $L = D^{-1}A$7, while for leaky-ReLU, the entire interval $L = D^{-1}A$8 is attainable. Thus, by learning the appropriate bias in $L = D^{-1}A$9, GCS can ensure that deep GNNs neither oversmooth (all features align with $Z^{(k)} = L^k Z$0) nor undersmooth (losing neighborhood coherence).

The SCT does not increase the norm of non-smooth features but permits dynamic redistribution of energy between smooth and non-smooth modes. This flexibility is essential for tasks on both homophilic and heterophilic graphs.

6. Empirical Evaluation and Practical Guidelines

Empirical studies across node-classification benchmarks (CORA, Citeseer, PubMed, Amazon Photo, Amazon Computers, Coauthor CS, Coauthor Physics, OGBN-arXiv, Chameleon, Squirrel, Web-KB) and multiple GNN backbones (GCN, ChebGCN, GraphSAGE, GAT, EGNN, GCNII, and others) establish:

• MADReg and AdaEdge: Statistically significant improvements in node classification accuracy (typically +0.4 to +1.4 points, occasionally more) and increase in MADGap for deeper models. The effect is most pronounced when standard GNNs suffer over-smoothing as layers increase (>4 layers) (Chen et al., 2019).
• SCT in GCNs: Augmented GCNs with SCT avoid oversmoothing collapse up to 32 layers and exhibit 1–10 percentage point accuracy gains on deep architectures. Feature smoothness, measured by $Z^{(k)} = L^k Z$1, is maintained in an optimal task-adapted regime, not forced to 1 as in vanilla GCN (Wang et al., 2024).

Recommended practices for GCS include:

• Monitor MAD, MADGap, and/or spectral smoothness per layer throughout training.
• Adjust $Z^{(k)} = L^k Z$2 (number of smoothing layers), the SCT/MLP regularization capacity, or polynomial filter coefficients based on performance and observed smoothness trajectories.
• Tune regularization ($Z^{(k)} = L^k Z$3) and smoothing depth jointly, as they interact nontrivially.
• Employ spectrum-aware filters or SCT-based smoothing on heterophilic graphs.

7. Impact, Limitations, and Future Directions

GCS unifies spectral analysis, geometric characterization of nonlinearities, and empirical regularization into a task-adaptive approach to graph feature smoothing. It reconciles the necessity of denoising and neighborhood information propagation with the risk of representational collapse.

However, there are important caveats:

• If task-relevant structure lies in low-variance spectral directions (as in extreme heterophily), even modest smoothing may be detrimental. In such cases, GCS must be restricted via depth, tailored polynomial filters, or augmented with non-mean aggregation mechanisms.
• The learnable biases (SCT) and adaptive regularization introduce moderate computational and tuning overhead, though these are reported to be minor ($Z^{(k)} = L^k Z$4 for connected-component eigenvectors) (Wang et al., 2024).
• The theory is most developed for linear and simplified message-passing rules; extensions to intricate spatial or attention-based GNNs remain an evolving area.

A plausible implication is that future GCS strategies may incorporate dynamic, data-driven monitoring and active control of spectral properties in complex GNN pipelines, potentially blending with automated edge editing and self-supervised regularization to further guard against oversmoothing across diverse graph types.