GCL-GCN: Enhanced Attributed Graph Clustering

• GCL-GCN is a deep graph clustering network that integrates global attention from Graphormer with local message aggregation from GCN and a contrastive learning module.
• It fuses attribute reconstruction and centrality-aware encodings to generate robust node embeddings for heterogeneous, sparse graphs.
• Experimental evaluations demonstrate significant clustering metric improvements over prior methods on benchmarks like Cora, ACM, and DBLP.

GCL-GCN (Graphormer and Contrastive Learning Enhanced Attributed Graph Clustering Network) is a deep attributed graph clustering architecture that integrates global attention via Graphormer, local message aggregation with GCN, and representation discrimination through a novel contrastive learning scheme. Designed for clustering tasks on complex sparse and heterogeneous graphs, GCL-GCN addresses critical limitations of conventional GCN and fusion-based graph clustering models by leveraging structural and attribute centrality encodings and contrastive pre-training to jointly enhance both global and local dependencies in node embeddings (Li et al., 25 Jul 2025).

1. Architectural Overview

GCL-GCN is a multi-branch, end-to-end unsupervised clustering network comprising four principal components: an Auto-Encoder (AE) for attribute reconstruction, a GCN encoder for local neighborhood aggregation, a Graphormer encoder for global multi-head self-attention enriched by centrality and spatial awareness, and a contrastive learning module acting as a feature-boosting pretext task. Each module operates either in succession or parallel, with their outputs fused for downstream clustering.

The processing pipeline is as follows:

1. The node feature matrix $X$ is encoded via AE to obtain a low-dimensional embedding $Z_{AE}$.
2. A graph augmentation (random feature masking) forms $X_{aug}$, which, along with $X$, is processed by a 2-layer GCN to produce $C_1$ and $C_2$. Contrastive loss on $(C_1,C_2)$ yields the discriminative feature matrix $X_c$.
3. The concatenated features $[X\|\|X_c]$ are separately injected into the AE, GCN, and Graphormer encoders, producing $Z_{\rm AE}$, $Z_{AE}$0, and $Z_{AE}$1.
4. A Representation Enhancement module incorporates the AE’s hidden states into each GCN and Graphormer layer.
5. The fused embedding $Z_{AE}$2 is clustered via soft student’s-t assignments.

The loss function jointly optimizes feature and structure reconstruction, contrastive discrimination (pre-training), and self-supervised clustering via KL divergence.

2. Key Mathematical Components

2.1 Graphormer Encodings

Centrality information is explicitly embedded:

• Degree centrality: $Z_{AE}$3
• Betweenness centrality: $Z_{AE}$4, quantifying mediated traffic through $Z_{AE}$5
• Closeness centrality: $Z_{AE}$6
• The centrality vector $Z_{AE}$7 is concatenated with node features.

Spatial encoding uses feature-space distances, $Z_{AE}$8. The Graphormer attention mechanism combines linear projections of features and centrality, producing the attention logit:

$Z_{AE}$9

with per-head softmax and output aggregation. Multi-head outputs yield $X_{aug}$0.

2.2 Contrastive Pre-Training

Contrastive loss is computed on the two GCN-based views:

1. Similarity function: $X_{aug}$1 combines cosine and Euclidean similarities.
2. For temperature $X_{aug}$2, the per-node loss is

$X_{aug}$3

2.3 Clustering and Reconstruction

• Clustering uses soft student’s-t assignments and sharpens them via a target distribution.
• Auto-Encoder and GCN/Graphormer decoders reconstruct both $X_{aug}$4 and adjacency matrix $X_{aug}$5.
• Consistency and clustering losses combine as $X_{aug}$6.

3. Training Procedure

The training protocol comprises a pre-training and a joint fine-tuning phase:

• Pre-training: The AE is first trained stand-alone; K-means on its bottleneck gives initial cluster centroids. The GCN-based contrastive module then discriminatively pre-trains features between $X_{aug}$7 and $X_{aug}$8, generating $X_{aug}$9.
• Joint Fine-tuning: The concatenated representations $X$0 are processed by AE, GCN, and Graphormer encoders, with AE hidden states injected into each layer of the GCN and Graphormer. The final fused embedding is obtained and clustering proceeds via soft assignments, backpropagating the unified loss. The process continues until convergence or early-stopping on the clustering objective.

The representation flow is: $X$1 GCN/Graphormer $X$2.

4. Implementation and Hyperparameters

Key configuration and implementation details include:

• Three-layer encoder/decoder architectures empirically perform best (layer-depth ablation).
• Hidden dimensions: $X$3 ($X$4 dataset-specific).
• Contrastive GCN: 2 layers, configurable dropout rate $X$5.
• Fusion coefficients $X$6 adapt to dataset, constrained to sum to 1.
• Loss weights: clustering $X$7; consistency $X$8.
• Optimization: Adam optimizer with learning rate $X$9 to $C_1$0, weight decay omitted, full-graph batch.
• Implementation uses PyTorch, tested on NVIDIA GTX 3070 with 32GB RAM.

Pre-training AE typically for 50 epochs stabilizes subsequent optimization. Best fusion weights and loss coefficients vary per graph; for example, Cora and Reuters prefer higher $C_1$1 (Graphormer emphasis), whereas DBLP prioritizes GCN.

5. Experimental Evaluation

GCL-GCN is benchmarked on six datasets—three citation networks (Cora, Citeseer, DBLP), ACM, and two non-graph benchmarks (HHAR, Reuters). Clustering metrics used include Accuracy (ACC), Normalized Mutual Information (NMI), Adjusted Rand Index (ARI), and F1-score.

• On Cora: ACC = 73.24% (+4.94 over best baseline), NMI = 55.16% (+13.01%), ARI = 52.19% (+10.97%), F1 = 64.79%.
• On ACM: ACC = 93.39% (+4.38%), with strong NMI and ARI.
• GCL-GCN consistently exceeds advanced baselines such as TDCN (Transformer-based) and the contrastive learning system CONVERT by 1–2 percentage points on all major datasets.
• Ablation experiments show removing the GCN branch yields a major drop (e.g., ACM ACC falls from 0.9382 to 0.7634), removing Graphormer results in a 2–4% accuracy loss, and dropping the contrastive module slightly (0.1–0.5% ACC), suggesting the greatest performance gain emerges from combining all modules.

6. Ablation, Hyperparameter, and Representation Analysis

Ablation studies on layer depth, encoding choices, and fusion weights demonstrate:

• Three-layer architectures outperform both shallower and deeper variants; four layers overfit, one or two layers underfit.
• Full centrality encoding (degree, betweenness, closeness) plus Euclidean spatial encoding yields the highest clustering quality.
• Fusion coefficients are dataset-dependent, with some tasks relying more on global (Graphormer), others on local (GCN) or attribute (AE) information.
• Heat-map and sensitivity analyses indicate relatively low sensitivity of overall performance to clustering and consistency loss weights.
• Simpler contrastive modules often outperform complex augmentations, suggesting robustness to augmentation-induced feature noise.
• t-SNE visualizations in the primary literature reveal better cluster separability with active Graphormer attention and contrastive pre-training.

7. Context, Innovation, and Impact

GCL-GCN introduces several innovations that distinguish it within deep graph clustering:

• Graphormer’s integration of attention-weighted centrality/spatial encodings captures global and local structure simultaneously, providing a richer message-passing mechanism than traditional GCNs.
• The contrastive learning module, employed in pre-training, yields more discriminative node features, reducing the risk of poor cluster formation due to attribute entanglement.
• By fusing AE’s attribute-driven representations into each GCN and Graphormer layer, the model ensures that attribute, local, and global cues are all available at every level of message-passing.
• The unified joint optimization of reconstruction, clustering, and distributional consistency objectives enables stable, robust training.
• Comprehensive empirical benchmarking shows significant performance gains across diverse graph types and attributes, with improvements of +4–13% over prior leading methods (Li et al., 25 Jul 2025).

Taken together, these design features make GCL-GCN a leading framework for attributed graph clustering, especially under conditions of sparsity and attribute heterogeneity.