Bridge the Gaps: Heterogeneous Attributed Graph Clustering via Quaternion Representation Learning

Abstract: Attributed graph clustering partitions nodes by jointly exploiting node attributes and graph topology. It remains challenging due to attribute heterogeneity and representation degradation during graph learning. Real-world datasets often contain heterogeneous attributes, i.e., numerical and categorical attributes, complicating unified representation learning. This challenge becomes more complex in attributed graphs, where constructing a clustering-friendly graph structure from attributes and topology remains difficult. Under deep graph architectures, repeated graph propagation causes node embeddings to become overly similar, leading to the over-smoothing (OS) effect. Meanwhile, graph representation learning amplifies topological influence, making discriminative attribute information harder to exploit for clustering, an effect we refer to as over-dominating (OD). To bridge these gaps, an end-to-end framework, Any-type attributed Graph REpresentation lEarning (AGREE), is proposed. It unifies attributed graphs and any-type attributed data through multi-level alignment and similarity-based graph construction. Quaternion-based graph convolution strengthens attribute interaction to alleviate OD, while shallow graph architectures help relieve OS. The learned embeddings are jointly optimized for graph reconstruction and clustering, without requiring a predefined number of clusters during training. Experiments on diverse benchmarks show that AGREE achieves strong overall performance in accuracy, robustness, and adaptability.

• The paper introduces AGREE, a framework that uses quaternion representation to mitigate over-smoothing and over-dominating effects in attributed graph clustering.
• It implements multi-level attribute alignment to fuse categorical and numerical features into a unified hyper-complex space for robust clustering.
• Empirical results on standard benchmarks demonstrate AGREE's resilience to topological noise, yielding superior ARI scores and enhanced cluster separability.

Quaternion-Based Representation Learning for Heterogeneous Attributed Graph Clustering

Introduction and Problem Formulation

The paper "Bridge the Gaps: Heterogeneous Attributed Graph Clustering via Quaternion Representation Learning" (2606.23199) presents AGREE, an end-to-end framework for attributed graph clustering. The work addresses two primary challenges: the integration of heterogeneous node attributes (categorical and numerical) and the mitigation of representation degradation phenomena, namely over-smoothing (OS) and over-dominating (OD), during graph neural network–based clustering.

While prior methods efficiently fuse attribute and topological information, they typically struggle with attribute heterogeneity and the construction of effective, clustering-friendly graph structures. Furthermore, repeated propagation in deep GNNs can induce OS—where node embeddings become indiscriminately similar—and attribute signals often get suppressed by the graph topology, leading to OD. AGREE is introduced as a solution, leveraging quaternion graph encoding to enhance attribute interaction in hyper-complex space, while modular multi-level alignment and unified graph construction ensure semantics across heterogeneous attributes are preserved and exploited.

Figure 1: Overview of AGREE’s motivation—multi-level aligned encoding unifies mixed-type attributes, and quaternion graph encoders strengthen cross-view interactions to alleviate over-smoothing and over-dominating effects.

AGREE Framework and Methodology

Multi-Level Attribute Alignment and Attributed Graph Construction

AGREE begins by harmonizing heterogeneous attribute spaces via four types of encoding:

• Value-level encoding: Captures the occurrence probability distribution for each categorical attribute value.
• Feature-level encoding: Encodes conditional dependency structures among categorical features, capturing attribute co-occurrence statistics.
• Attribute-type encoding: Projects categorical values onto one-dimensional metric spaces using conditional dependencies, thus bridging attribute type disparities and enabling homogeneous distance computation with numerical attributes.
• Object-level encoding: Establishes cross-object similarity via a fully-connected graph using a unified, attribute-type-aware dissimilarity metric.

These coupled encodings yield a unified, information-rich representation, facilitating object-wise similarity computation and subsequent attributed graph construction.

Figure 2: AGREE framework—mixed-type attributes are aligned and organized into an attributed graph, projected into hyper-complex (quaternion) space for joint graph/attribute learning via quaternion graph convolution.

Quaternion Representation Learning and Four-View Projection

Unlike methods operating in real-valued spaces, AGREE introduces a projection mechanism transforming encoded attributes into four independent views, organized into quaternion-valued features. Each node’s representation is constructed as a quaternion $$\mathbf{F} = F_r + F_x \mathbf{i} + F_y \mathbf{j} + F_z \mathbf{k}$$, providing four times the degrees of freedom at the same parameter scale compared to real-valued encoders. Feature-level interaction occurs via the Hamilton product, supporting efficient, structured cross-view interactions and mitigating the OD effect by enhancing the preservation and integration of attribute information even under strong topological signals.

A shallow quaternion graph encoder then propagates these representations, combining hyper-complex attribute interactions with structural graph information, mitigating OS by constraining the homogenization of node embeddings typically observed in deep stackings of real-valued GCN layers.

Unified Objective and Clustering

AGREE is optimized with a composite loss: graph reconstruction loss (KL divergence), spectral clustering loss (trace minimization over the Laplacian of reconstructed graph adjacency), and regularization. Importantly, the model does not require knowledge of the number of clusters during training, enhancing its practical applicability to real-world clustering tasks with unknown $k$.

Empirical Analysis

Robustness to Topological Perturbations

The paper conducts controlled perturbation studies on standard graph benchmarks (e.g., ACM, CORA, AMAP), where random cross-class edges are introduced. AGREE’s quaternion-based encoder demonstrates substantially higher alignment between original and perturbed representations compared to real-valued GCN counterparts, indicating robustness to spurious topological influence and a weaker OD effect.

Figure 3: Embedding alignment between original and perturbed graphs—AGREE yields higher consistency relative to real-valued GCNs, indicating improved robustness against topological noise.

Clustering Performance Under Topological Noise

ARI scores measured under increasing perturbation ratios further highlight AGREE’s stability, with slower degradation as noise increases, corroborating the resilience of quaternion-based attribute interaction against graph irregularities.

Figure 4: Clustering quality (ARI) under varying perturbation ratios—AGREE preserves clustering fidelity better than real-valued encoders under structural noise.

Efficacy of Multi-Level Alignment

Ablation studies isolating value-level, feature-level, and attribute-type alignments, as well as object-level graph construction strategies, confirm the necessity of multi-level and attribute-type-aware encodings. Replacing proposed object-level construction with one-hot similarity graphs consistently lowers performance, affirming the effectiveness of the AGREE alignment pipeline.

Figure 5: ARI for progressive alignment ablation on selected datasets—complete multi-level alignment with AGREE’s graph achieves the highest performance.

Qualitative Representation Analysis

$t$-SNE projections of AGREE embeddings on the ACM dataset exhibit tight intra-cluster compactness and clear inter-cluster separation, with higher NMI compared to alternative approaches, visually confirming superior embedding quality for clustering.

Figure 6: Visualization of node embeddings on ACM via $t$-SNE—AGREE delivers improved cluster separability and compactness.

Discussion and Implications

The introduction of quaternion-based graph encoding extends hyper-complex representation learning from supervised tasks—where inherent multi-tuple data (e.g., color channels, 3D pose) is common—into the unsupervised, heterogeneous attributed clustering domain. Theoretical analysis and experiments reveal:

• Quaternion GCNs provide increased degrees of freedom and richer feature co-interaction, crucial for preserving attribute semantics in the presence of strong or noisy graph topology.
• Shallow architectures enabled by expressive quaternion representation mitigate OS without requiring deep propagation, maintaining cluster-discriminative embeddings.
• Multi-level alignment strategies generalize beyond conventional one-hot or statistical encodings, capturing dependence structure and supporting seamless handling of mixed-type data.
• AGREE operates without explicit supervision of the cluster number, making it deployable in unsupervised and open-world scenarios.

In practical terms, AGREE’s robustness to both topological and attribute heterogeneity motivates application to settings such as IoT device clustering, biological network analysis, and mixed-typed social graph mining. Its attribute interaction mechanism is particularly relevant for graphs where topology is unreliable, incomplete, or entirely induced from attribute similarity.

Future Prospects

Future conceptual expansions may target: richer hyper-complex extensions (e.g., octonion representations), adaptive cluster number prediction, federated or decentralized heterogeneous graph clustering, and bias-mitigated representation strategies for extreme attribute imbalance or skew. Deep integration with auxiliary modalities or explicit external semantic priors remains an open direction for large-scale attributed graph learning.

Conclusion

AGREE advances attributed graph clustering via a systematic, hyper-complex approach, unifying attribute heterogeneity through multi-level encoding and leveraging quaternion-based graph encoders to robustly integrate structure and attribute semantics. This work provides both an effective practical tool and a methodological extension of hyper-complex representation learning, setting the stage for further research in unsupervised attributed graph analysis (2606.23199).