Disentangle-then-Refine: LLM-Guided Decoupling and Structure-Aware Refinement for Graph Contrastive Learning

Abstract: Conventional Graph Contrastive Learning (GCL) on Text-Attributed Graphs (TAGs) relies on blind stochastic augmentations, inadvertently entangling task-relevant signals with noise. We propose SDM-SCR, a robust framework anchored in Approximate Orthogonal Decomposition. First, the Semantic Decoupling Module (SDM) leverages the instruction-following capability of LLMs to actively parse raw attributes into asymmetric, task-oriented signal and noise views. This shifts the paradigm from random perturbation to semantic-aware disentanglement. Subsequently, Semantic Consistency Regularization (SCR) exploits the spectral observation that semantic signals are topologically smooth while residual noise is high-frequency. SCR functions as a selective spectral filter, enforcing consistency only on the signal subspace to eliminate LLM hallucinations without over-smoothing. This ``Disentangle-then-Refine'' mechanism ensures rigorous signal purification. Extensive experiments demonstrate that SDM-SCR achieves SOTA performance in accuracy and efficiency.

• The paper presents SDM-SCR, a two-stage framework that leverages LLM-driven semantic decoupling to separate task-relevant signal from noise.
• It integrates semantic parsing with spectral refinement, using selective regularization to enhance node embedding discriminability.
• Empirical studies show that SDM-SCR achieves state-of-the-art node classification across diverse datasets, outperforming stochastic augmentation methods.

Disentangle-then-Refine: LLM-Guided Signal Decoupling and Spectral Refinement in Graph Contrastive Learning

Introduction

Text-attributed graphs (TAGs) are central in applications where the interplay between structured topology and rich node-level text underpins complex reasoning tasks. While graph neural networks (GNNs) have achieved notable impact on such domains, the persistent challenge in unsupervised and self-supervised learning paradigms (notably, graph contrastive learning, GCL) is the synthesis of augmentation schemes that robustly expose task-dependent semantics while abating nuisance variation and noise. Prevailing approaches typically exploit stochastic feature perturbations or graph topological modifications, but these methods are fundamentally agnostic to the semantic granularity demanded by downstream objectives, leading to a detrimental entanglement of discriminative signal with irrelevant patterns.

"Disentangle-then-Refine: LLM-Guided Decoupling and Structure-Aware Refinement for Graph Contrastive Learning" (2604.14746) introduces SDM-SCR, a pipeline that directly addresses the insufficiencies of blind data augmentation in TAG-based contrastive representation learning. Leveraging instruction-tuned LLMs as semantic decomposers, this framework implements a two-stage approach: (1) task-aware semantic decoupling, and (2) structure-aware semantic consistency regularization, bridging the semantic abstraction capabilities of LLMs with spectral graph theoretical insights.

LLM-Guided Decoupling: The Semantic Decoupling Module (SDM)

Traditional GCL methods such as GRACE, MVGRL, and ProGCL construct views via random feature masking and edge dropping, which are indifferent to the task-dependent nature of semantic information. The SDM module in this work fundamentally departs from such augmentation schemes. It leverages an instruction-following LLM to semantically parse each node's textual attribute into two asymmetric components: one dominant in task-relevant signal, the other in noise. This operationalizes an approximate orthogonal decomposition (AOD) within the node embedding space, where the signal ($\mathbf{s}_i$) and noise ($\mathbf{n}_i$) are minimally correlated; the decomposition is semantically anchored by explicit task instructions rather than stochastic transformation.

The SDM is seamlessly pluggable into existing GCL frameworks. Rather than modifying encoder architectures, it substitutes upstream feature representations with LLM-refined signal/noise views, thereby repositioning the role of augmentation from blind perturbation to semantically controlled view generation.

Figure 1: Example of SDM semantic parsing on Ele-Photo TAG—task-relevant and irrelevant text segments are explicitly highlighted per downstream classification objective.

Figure 2: Contrasting SDM’s LLM-guided semantic view construction with conventional random augmentation in GCL.

A critical advantage of SDM is its use of asymmetric contrastive optimization. The method uses the LLM-generated signal view as the positive anchor and the noise view (from different nodes) as negatives, thus explicitly pushing original embeddings toward the instruction-driven semantic subspace while repulsing the dominant nuisance directions. This strategy notably increases the purity and discriminability of the learned representations.

Structure-Aware Spectral Refinement: Semantic Consistency Regularization (SCR)

While LLMs are highly effective as semantic decomposers, they are error-prone and may leak residual hallucinations into the relevant signal view. SCR addresses these limitations via a spectral approach aligned with graph signal processing theory. The principal insight: bona fide semantic signals exhibit topological smoothness (are low-frequency with respect to the graph Laplacian), whereas hallucinations and task-irrelevant factors are typically high-frequency and topologically inconsistent.

SCR enforces the following:

• Consistency is selectively imposed on the relevant semantic subspace between structurally connected nodes, thereby acting as a localized low-pass spectral filter exclusively for the LLM-derived signal view.
• The noise subspace is left unconstrained, serving as a spectral sink, so that no high-frequency components are inadvertently mixed into the semantic propagation process.

Under this regime, instance-level decoupling cleans the embeddings, and SCR further purifies signal propagation by suppressing LLM-induced high-frequency residuals through variance reduction mechanisms inherent to aggregation across neighborhoods. This kind of spectral gating obviates the classic “over-smoothing” issue endemic to deeper GNNs and prevents negative transfer in heterogeneous or low-homophily regions.

Empirical Results and Analysis

Extensive benchmarking across canonical datasets (Citeseer, Pubmed, Wiki-CS, Ele-Photo, Books-History) substantiates several strong claims:

• The SDM-SCR pipeline achieves SOTA node classification accuracy on all evaluated datasets, surpassing both stochastic and prior LLM-augmented GCL baselines. Performance is retained even when instantiated with parameter-lean open-source LLMs (e.g., Gemma-3-1B), evidencing the minimal reliance of effectivity on LLM model scale.
• Using SDM as a plug-in augmentation on various GCL backbones consistently raises performance, distinguishing signal-dominant embeddings that are universally more robust than the entangled outputs of stochastic augmentation methods.
• Ablation experiments establish that the contrastive repulsion between original and noise views (the negative sample effect of SDM) is indispensable. Merely adopting LLM signal embeddings as features yields inferior results; the full SDM-SCR approach realizes the benefits of both semantic alignment and explicit denoising.
• SCR’s selective spectral regularization improves upon both “blind” smoothing and naive LLM regularization, yielding error bounds provably lower than classic spatial message passing, even under heterophilic graphs.

Theoretical and Practical Implications

This framework advances the integration of language-driven semantic inferencing with geometric deep learning and self-supervised graph representation. Theoretically, the implementation of approximate orthogonal decomposition at the semantic-structural interface operationalizes the longstanding desideratum of disentanglement in unsupervised learning for TAGs, and the regularization by selective spectral filtering tightly aligns with optimal signal-processing principles for graph domains.

Practically, SDM-SCR motivates more systematic exploitations of task-specific instruction-tuned LLMs in automated feature engineering for complex multimodal graph data. Its pluggability and efficiency open up cost-effective pathways for scaling GCL to large TAGs without incurring prohibitive inference costs. Furthermore, the work demonstrates that strong semantic decoupling can be realized even with small and open LLM backbones, democratizing the applicability of this paradigm.

The limiting factor remains the residual imperfection of LLM-based semantic parsing—however, the empirical evidence here suggests that LLM errors are sufficiently uncorrelated with graph topology to be effectively suppressed by spectral regularization, providing robustness against negative transfer and over-smoothing.

Conclusion

SDM-SCR represents a substantive refinement to the GCL paradigm for text-attributed graphs. By embedding LLM instruction-driven semantic parsing at the data augmentation phase and enforcing spectral denoising of the resultant signal subspace, it enables the construction of discriminative, semantically pure node embeddings suitable for a wide variety of graph representation learning tasks. As LLM technology advances, further gains in semantic decomposition quality can be harnessed. The methodological unification of orthogonal decomposition and spectral graph regularization chart promising directions for robust, generalizable self-supervised graph learning.