Cluster-wise Graph Analysis

• Cluster-wise graphs are structures that explicitly model nodes and edges based on clusters, enabling clear demarcation of intra-cluster similarity and inter-cluster separation.
• They integrate joint optimization of graph topology and clustering assignments to enhance methods like graph pooling, spectral clustering, and multiscale representations.
• Their applications span graph neural networks, statistical inference, and efficient computation, yielding improved clustering accuracy and computational performance.

A cluster-wise graph is a graph-based structure in which nodes, edges, or higher-order relations are organized, inferred, or modeled explicitly with respect to clusters—sets or groups of entities exhibiting high intra-group similarity and, typically, lower inter-group similarity. Across disparate subfields, the cluster-wise graph concept encompasses both the construction of graphs to faithfully encode cluster structure, and the use of clustering as an inductive bias for learning, inference, or efficient computation on graphs. Primary use cases include clustering-aware graph learning, coarsening, pooling, graph transformers, ensemble or consensus clustering, statistical inference for clustered variables, and random graph models that mimic empirical clustering phenomena.

1. Optimization Frameworks for Cluster-wise Graph Learning

Cluster-aware graph learning methods focus on constructing a similarity graph whose structure directly supports or reflects the desired clustering, often via joint optimization of the graph topology and the cluster assignment matrix.

• Joint Graph and Clustering Optimization: In "Clustering-aware Graph Construction: A Joint Learning Perspective," the similarity matrix $Z$ and cluster indicator matrix $F$ are learned together by minimizing:

$$\min_{Z,F} \; \alpha\,\|X - XZ\|_F^2 + \|Z - FF^T\|_F^2 + \beta\,\|Z - W\|_F^2,$$

subject to $Z \ge 0$, $\mathrm{diag}(Z)=0$, $F \ge 0$ (Jia et al., 2019). The key coupling term $\|Z - FF^T\|_F^2$ regularizes $Z$ to a block-diagonal structure matching cluster assignments. Multiplicative updates are used for both $Z$ and $F$, with provable decrease of objective and block-diagonal convergence in $F$0.

• Rank and Spectral Constraints: Methods such as SGSK (Kang et al., 2020) and SPC (Kang et al., 2019) directly impose spectral or rank constraints to ensure the learned similarity matrix $F$1 (or $F$2) induces exactly $F$3 connected components (clusters). In particular, penalizing the sum of the $F$4 smallest Laplacian eigenvalues ensures the cluster-wise graph structure:

$F$5

with $F$6 collecting cluster indicators as Laplacian eigenvectors (Kang et al., 2020).

• Kernel and Similarity Preservation: The SPC framework constrains $F$7 to be similarity-preserving relative to an initial kernel $F$8, via a term $F$9, and forces cluster-wise (block-diagonal) connectivity by embedding the first $$\min_{Z,F} \; \alpha\,\|X - XZ\|_F^2 + \|Z - FF^T\|_F^2 + \beta\,\|Z - W\|_F^2,$$0 Laplacian eigenvectors in $$\min_{Z,F} \; \alpha\,\|X - XZ\|_F^2 + \|Z - FF^T\|_F^2 + \beta\,\|Z - W\|_F^2,$$1 (Kang et al., 2019). The multi-kernel variants (SGMK, mSPC) simultaneously learn optimal kernel weights alongside the cluster-wise graph (Kang et al., 2020, Kang et al., 2019).

In all these settings, the learned cluster-wise graph is not a static data structure but a flexible variable determined by direct integration with clustering objectives. This approach leads to graphs whose component structure provides both inferential and computational benefits for downstream clustering and labeling.

2. Hierarchical and Multiscale Cluster-wise Graphs

Hierarchical or multiscale constructions involve recursively merging (or splitting) clusters to expose relationships between clusters at various resolutions.

• Multiscale Graph Construction: "Multiscale Graph Construction Using Non-local Cluster Features" formalizes multiscale cluster-wise graphs as sequences of graphs $$\min_{Z,F} \; \alpha\,\|X - XZ\|_F^2 + \|Z - FF^T\|_F^2 + \beta\,\|Z - W\|_F^2,$$2 at different scales, where each node at level $$\min_{Z,F} \; \alpha\,\|X - XZ\|_F^2 + \|Z - FF^T\|_F^2 + \beta\,\|Z - W\|_F^2,$$3 corresponds to a cluster from the previous (finer) level (Kaneko et al., 2024). The multi-step process consists of:
  1. Extracting feature representations for each cluster using intra-cluster $$\min_{Z,F} \; \alpha\,\|X - XZ\|_F^2 + \|Z - FF^T\|_F^2 + \beta\,\|Z - W\|_F^2,$$4-means.
  2. Computing optimal transport (OT) distances between clusters, viewing their features as discrete measures.
  3. Building a variable-$$\min_{Z,F} \; \alpha\,\|X - XZ\|_F^2 + \|Z - FF^T\|_F^2 + \beta\,\|Z - W\|_F^2,$$5 nearest neighbor graph (VkNNG) on clusters using OT-based similarities, followed by spectral clustering to obtain coarser clusters.

This methodology supports non-local merging: clusters with similar internal feature distributions may be grouped regardless of graph distance.

• Hierarchical Pooling in GNNs: Local Cluster Pooling (LCPool) implements cluster-wise graph pooling by scoring and selecting clusters, pooling node features, and constructing edges between pools based on multi-hop neighborhood overlap (Chen, 2024). This approach is grounded on local, node-centered clusters and enables adaptive, structure-aware coarsening.
• Generalizing Lloyd's Algorithm: The generalized Lloyd approach (Zaman et al., 2023) adapts $$\min_{Z,F} \; \alpha\,\|X - XZ\|_F^2 + \|Z - FF^T\|_F^2 + \beta\,\|Z - W\|_F^2,$$6-means to graphs by defining clustering energy in terms of shortest-path distances, maintaining cluster connectivity, and rebalancing clusters for size and "well-centeredness." This ensures cluster-wise graph partitioning suitable for tasks such as algebraic multigrid aggregation.

These algorithms operationalize cluster-wise graphs as flexible hierarchical or coarsened representations, facilitating tasks such as segmentation, pooling, or multi-resolution analysis.

3. Cluster-wise Graphs for Graph Neural Networks and Transformers

The explicit modeling of node clusters and their relations provides a powerful inductive bias for GNN architectures and graph transformers.

• Cluster-wise Graph Transformer (Cluster-GT): Recent architectures, e.g., (Huang et al., 2024) and DeCoda (Liang et al., 30 Jul 2025), introduce dual-granularity cluster-wise graphs. Nodes are assigned to clusters using METIS or similar partitioning, forming a cluster-assignment matrix. The coarsened "cluster graph" adjacency is $$\min_{Z,F} \; \alpha\,\|X - XZ\|_F^2 + \|Z - FF^T\|_F^2 + \beta\,\|Z - W\|_F^2,$$7. In Cluster-GT, information flows via Node-to-Cluster Attention, which fuses node and cluster representations using a bi-level kernel—either via a tensor-product or convex combination—and ensures that both node-level and cluster-level information shape each attention update. The architecture alternates self-attention at node and cluster levels, with cross-attention transmitting information between these granularities (Huang et al., 2024, Liang et al., 30 Jul 2025).
• Differentiable Cluster-GNNs and Bipartite Models: DC-GNN (Dong et al., 2024) augments the node set with learnable cluster-nodes (both global and local) and formulates learning as an entropy-regularized optimal transport assignment between nodes and clusters. The resulting structure is a bipartite cluster-wise graph on which all message passing can be implemented with differentiable updates alternating between Sinkhorn normalization of cluster assignment and embedding updates. This provides a computational mechanism for enhancing both local aggregation (robust to heterophily) and long-range signal propagation.
• Cluster-wise Graphs in Pooling and Representation: Hierarchical pooling mechanisms, such as LCPool (Chen, 2024), generalize classic pooling by explicitly modeling node clusters as the units of coarsening, rather than simple node selection or dense assignments. Cluster-wise edge constructions encode the overlap of local neighborhoods up to graph distance three, preserving higher-order connectivity crucial for effective GNN information propagation.

The integration of cluster-wise graphs within GNNs and transformers reflects the growing recognition that both fine-grained (node-level) and coarse-grained (cluster-level) structure must be harmonized for scalable, expressive graph representations.

4. Cluster-wise Graphs in Statistical Inference and Ensemble Clustering

Cluster-wise graphs also occur as objects of inference in high-dimensional statistics and as tools for meta-level relation modeling in ensemble and consensus clustering.

• Cluster-Based Graphical Models: Inference for cluster-average and latent-variable graphs (Eisenach et al., 2018) begins by clustering variables/features using specialized algorithms (e.g., PECOK), then performing inference on the estimated conditional independence (CI) graph among cluster-averaged variables or latent factors. Precision matrix estimation, debiasing, and FDR-controlled edge selection are all performed in a cluster-wise regime, with explicit account for clustering uncertainty and theoretical guarantees for Berry-Esseen bounds.
• Ensemble and Consensus Clustering: Cluster-wise graphs serve as meta-level entities in ensemble clustering, as in the ECPCS framework (Huang et al., 2018). Here, base clusters from multiple clusterings form the nodes of a "cluster similarity graph," with edges weighted by measures such as the Jaccard index. Multi-step random-walk propagation on this graph captures multi-scale similarity, which is then used to enhance object-level co-association matrices or to partition clusters into meta-clusters, integrating both direct and indirect, multi-path relationships between clusters for robust consensus clustering.
• Random Graph Models with Cluster Structure: Generalized random graph models, such as the scalable ensembles based on the Gleeson algorithm (Wang, 2013), explicitly parameterize motifs (e.g., shared-edge triangles, k-cliques, structural holes) to realize random graphs with prescribed numbers of edges, nodes, and (critically) triangles. These cluster-wise random graphs quantitatively reproduce empirical network transitivity/clustering, while preserving maximum randomness at higher-order motif scales.

These models and techniques confirm the centrality of the cluster-wise graph abstraction both as an inference target and as a representational backbone for interpreting complex, high-dimensional, or ensemble-derived structures.

5. Applications, Empirical Performance, and Theoretical Guarantees

Cluster-wise graph representations have demonstrated empirical and theoretical benefits across clustering, classification, link prediction, and computational acceleration.

• Clustering and Classification: Structured graph learning with rank constraints and self-expressiveness yields consistent, state-of-the-art clustering and semi-supervised classification performance, outperforming spectral clustering, robust kernel $$\min_{Z,F} \; \alpha\,\|X - XZ\|_F^2 + \|Z - FF^T\|_F^2 + \beta\,\|Z - W\|_F^2,$$8-means, and other strong baselines across standard datasets (Kang et al., 2020, Jia et al., 2019). The combination of local and global structure, enforced block-diagonal graph learning, and kernelization underlies these empirical advances.
• Improved Graph Computations: Hierarchical clustering-based reordering and cluster-wise computation architectures for sparse matrix-matrix multiplication (SpGEMM) accelerate scientific workloads, achieving up to $$\min_{Z,F} \; \alpha\,\|X - XZ\|_F^2 + \|Z - FF^T\|_F^2 + \beta\,\|Z - W\|_F^2,$$9 average speedup compared to classic row-wise multiplication. These improvements stem from increased data reuse within cluster-wise access patterns and are robust to preprocessing cost and graph size (Islam et al., 28 Jul 2025).
• Graph Transformers and Pooling: Cluster-wise graph attention in modern transformers (Cluster-GT) delivers competitive or superior performance on graph-level prediction tasks (e.g., ZINC, MolHIV), with ablation studies showing tangible gains from explicit dual-granularity attention (Huang et al., 2024, Liang et al., 30 Jul 2025). Theoretical results demonstrate that node- and cluster-level kernels in attention outperform node- or cluster-only variants and specialize adaptively to graph domain.
• Large-scale Generation and Evaluation: Efficient algorithms for generating cluster-wise random graphs at million-node scale robustly preserve key metrics (clustering coefficient, path length distributions) while matching empirical networks in both triadic (triangle) and higher-order motif statistics (Wang, 2013).
• Statistical Inference: Cluster-based graphical models provide valid confidence intervals and hypothesis testing for CI graphs of clusters, incorporating the randomness of clustering and ensuring FDR control under minimal conditions (Eisenach et al., 2018).

Empirical evidence from these studies collectively validates the practical and theoretical value of constructing, learning, and reasoning on cluster-wise graphs.

6. Parameterization, Complexity, and Open Directions

The design of cluster-wise graph methods entails careful parameterization and complexity optimization:

• Overlap and Sensitivity: In pairwise-overlapping $Z \ge 0$0-means (Bauman et al., 2017), the degree of overlap (parametrized by $Z \ge 0$1 or $Z \ge 0$2) allows tuning the "edge density" of and sensitivity to inter-cluster adjacencies in the resulting cluster-wise graph.
• Spectral/Rank Constraints: The necessity of enforcing exact numbers of connected components (clusters) is met by spectral regularization or Laplacian rank constraints (e.g., via $Z \ge 0$3 or directly penalizing eigenvalues). The theoretical underpinning here is the direct correspondence between zero-eigenvalues and graph components.
• Assignment Algorithms: Blockwise bi-convexity, as in alternating optimization schemes, leads to iterative, convergent updates with polynomial complexity (with further reductions possible via sparsity). For instance, joint $Z \ge 0$4–$Z \ge 0$5 updates are cubic in $Z \ge 0$6 per iteration; Laplacian eigen-decompositions (for $Z \ge 0$7) are $Z \ge 0$8 but can be reduced for small $Z \ge 0$9.
• Cluster Granularity and Hierarchy: The number of clusters, assignment overlap, and granularity (fine/coarse, hard/soft partitions, degree of pooling) are central hyperparameters and can be cross-validated/data-driven, or adaptively end-to-end trained in modern GNNs and transformers.
• Empirical Complexity: Cluster construction (e.g., hierarchical clustering for SpGEMM) is carefully orchestrated to match or stay below the cost of core downstream computations (Islam et al., 28 Jul 2025).

Challenges remain in (i) efficiently learning soft or overlapping cluster assignments at scale, (ii) integrating differentiable cluster assignment into graph transformers and GNNs (extensions beyond METIS or hard partitioning), (iii) theoretical expressivity analysis of dual-granularity architectures, and (iv) extending cluster-wise constructions and inference to dynamic and attributed graphs.

---

In summary, the cluster-wise graph paradigm provides a foundational framework across graph learning, clustering, efficient computation, statistical inference, and random graph modeling. By structurally encoding clusters at multiple levels and integrating node- and cluster-level information flows, it enables finer control over representation, inference, and performance in graph-based tasks (Bauman et al., 2017, Jia et al., 2019, Kang et al., 2020, Huang et al., 2024, Dong et al., 2024, Huang et al., 2018, Liang et al., 30 Jul 2025, Kaneko et al., 2024, Wang, 2013, Eisenach et al., 2018, Chen, 2024, Islam et al., 28 Jul 2025, Zaman et al., 2023, Kang et al., 2019).