Multi-Coordinate Clustering

• Multi-coordinate clustering is a data analysis approach that integrates multiple, distinct feature spaces or latent coordinate systems to reveal diverse structural insights.
• It employs tensor algebra, independent subspace analysis, and Bayesian nonparametrics to achieve high clustering accuracy and robustness in high-dimensional, heterogeneous datasets.
• These methods enhance performance in applications such as image recognition, document categorization, and multi-omics analysis by balancing consensus and diversity across views.

Multi-coordinate clustering refers to a family of methodologies that construct clusterings by leveraging multiple, structurally distinct feature representations or latent coordinate systems. Approaches in this area address the limitations of traditional single-view clustering by explicitly modeling, inferring, or exploiting the multiplicity of coordinate systems—arising from multiple feature spaces, multi-modal data, tensor-valued structure, or latent views. Theoretical and algorithmic contributions in this field are unified by the objective of discovering either a consensus clustering operating across coordinates or, more generally, a set of alternative clusterings that are mutually non-redundant and expose different aspects of structure in the data. This article surveys the foundational models and recent advances in multi-coordinate clustering across tensor algebraic, subspace-based, multi-view, multimodal proxy, and Bayesian nonparametric perspectives.

1. Tensor and Algebraic Foundations for Multi-Coordinate Clustering

Tensor-based frameworks provide a principled mechanism for multi-coordinate clustering by operating directly on high-order array data, preserving intrinsic structure. The Sparse Submodule Clustering (SSmC) framework (Kernfeld et al., 2014) generalizes sparse subspace clustering (SSC) to multi-way arrays, using the t-product algebra for tensors of order three or higher. In the SSmC paradigm, each data sample is treated as an oriented matrix (mode-3 fiber or "tube"), and affinities are built via a sparse self-representation in a free module over the commutative ring of tubes, with multiplication defined by tube-wise circular convolution. This approach enables direct modeling of datasets such as images, videos, or multivariate time series without vectorization, maintains invariances such as shift, and supports theoretical guarantees: under linear independence or incoherence of submodules, the ℓ_{1,1,2}-minimization yields clustering solutions entirely within true submodules. Empirically, SSmC demonstrates improved robustness to noise and shift and superior performance on real-world image and digit recognition tasks relative to matrix-space SSC.

2. Subspace Decomposition and Multiple Clusterings

Multiple coordinate clusterings may be constructed by explicitly seeking statistically independent subspaces, each supporting its own clustering. The MISC (“Multiple Independent Subspace Clusterings”) method (Wang et al., 2019) decomposes the data matrix $X \in \mathbb{R}^{d \times n}$ via independent subspace analysis (ISA), generalizing ICA by grouping source components into subspaces that minimize inter-subspace mutual information. The number of subspaces is chosen via the minimum description length principle. Each subspace is then clustered via (kernelized, graph-regularized) semi-NMF, yielding multiple clusterings residing in non-redundant coordinates. This structure guarantees interpretability—different clusterings reflect genuinely independent aspects—and empirical results confirm superior recovery of ground-truth views on synthetic and real datasets.

Complementary to independent subspaces, "multi-view" frameworks such as DMJC (Lin et al., 2018) maintain parallel deep encoders for each view and optimize joint clustering losses under different fusion schemes (e.g., implicit via soft assignment, explicit via assignment fusion), learning complementary and consensus information.

3. Bayesian Multi-Coordinate Generative Modeling

Bayesian nonparametric models formalize multi-coordinate structure by treating views or coordinate systems as latent assignments, with the number of views, the subject and feature partitions per view, and relevant model parameters all learned from data. The multiple co-clustering model (Tokuda et al., 2015) posits a stick-breaking prior over views and cluster groups in each view, supporting block-wise modeling of heterogeneous data (Gaussian, Poisson, multinomial). The resulting variational inference yields, as a "multi-coordinate code," per-row and per-column cluster assignments for every view—allowing subject and variable groupings to vary flexibly and reflecting interpretable mechanisms such as feature selection unique to each coordinate system.

The Latent Simplex Position (LSP) model (Duan, 2019) leverages similarity matrices generated from multiple views, refining co-assignment probabilities via low-rank approximations (pairwise inner product of simplex coordinates) and selects from a reduced set of shared parameterizations, enabling dimension reduction and tractable uncertainty quantification over cluster assignments within each view.

4. Multi-View and Multimodal Clustering

Multi-view and multimodal clustering systems generalize multi-coordinate clustering to settings where each view may correspond to a different data acquisition process, modality, or feature set. The Dual-space Co-training Large-scale Multi-view Clustering (DSCMC) (Kong et al., 2024) introduces a co-training paradigm across original and latent spaces by learning per-view mapping matrices and a shared latent anchor graph, equipped with efficient optimizations (SVD-based updates, quadratic programming for anchor assignment). DSCMC’s alternating minimization under orthogonality and sparsity constraints enables robust, scalable clustering in heterogeneous and large-scale scenarios, with element-wise view-weighting further insulating against dominating or noisy views.

Recent advances such as Multi-Sub (Yao et al., 2024) extend model customization to the user level, aligning textual prompts denoting aspects of interest (e.g., color, shape) with the corresponding visual latent representation. This is achieved by leveraging proxy word-based subspace construction via LLMs (e.g., GPT-4), enabling dynamic definition and inference of clusterings for arbitrary user-specified coordinates or semantic axes.

5. Multiple Clustering Objectives: Diversity, Redundancy Control, and Consensus

A central challenge in multi-coordinate clustering is to ensure that alternative clusterings are both high quality and non-redundant. Methods such as DMClusts (Wei et al., 2019) employ deep matrix factorization across views, constructing multiple clusterings via layer-wise representations, and introduce a balanced redundancy term penalizing excessive overlap either by consistent co-assignment or consistent separation of pairs in different clusterings. This design produces orthogonal or otherwise decorrelated clustering solutions, measured both by internal clustering indices (Silhouette, Dunn) and redundancy metrics (NMI, Jaccard coefficient).

MVMC and MVMCC (Yao et al., 2019) decompose the self-representation matrix for each view into a commonality (shared) and individuality (per-clustering) component and enforce diversity by penalizing the Hilbert–Schmidt Independence Criterion (HSIC) between the individual matrices, while maintaining smoothness of the common matrix across view-specific graph Laplacians. Factorizations yield cluster labels for both samples and features.

6. Hierarchical and Large-Scale Multi-Coordinate Structures

For high-dimensional and large-scale scenarios, scalability and hierarchical organization become paramount. In isotropic dynamic hierarchical clustering (Sadikov et al., 2016), data are organized as a balanced tree of clusters, each represented by a center and radius in high-dimensional Euclidean space. O(1) incremental algorithms maintain sufficient statistics and bounding volumes; splits minimize overlap or radii, supporting robust pruning and efficient search. Sparse data representations and dynamic balancing strategies permit scalable processing of tens of millions of points.

Block-greedy coordinate descent (Scherrer et al., 2012) leverages feature clustering for algorithmic acceleration: features are partitioned into blocks to minimize cross-block inner product (maximal inter-block coherence). This not only improves parallel convergence rates for sparse learning problems but also connects multi-coordinate clustering to parallel optimization motifs.

7. Applications, Empirical Findings, and Extensions

Multi-coordinate clustering models have been empirically validated in diverse application domains: image and face databases (illumination, pose, and identity), document categorization, multi-omics biomedical data (features such as SNPs, methylation, psychometric scores), and multimodal visual/textual corpora. Across methods, unified findings include:

• Superior recovery of "ground truth" or domain-specific facets versus single-view or fully fused approaches, as measured by ARI, NMI, and other clustering indices (Kernfeld et al., 2014, Tokuda et al., 2015, Wei et al., 2019, Yao et al., 2024).
• Robustness to noise, missing data, and high-dimensional settings due to preserved coordinate structure and feature selection (Kernfeld et al., 2014, Tokuda et al., 2015).
• Flexibility in handling heterogeneous datatypes and integrating interpretability or user intent (Multi-Sub (Yao et al., 2024)).
• Theoretical guarantees (uniqueness, redundancy control, convergence rates) in both probabilistic and algebraic frameworks.

Extensions are actively pursued along axes of dynamic or streaming data, semi-supervised and constrained clustering, higher-way (tensor) generalizations, view-overlap or hierarchy, and increasingly, integrating attention or learned fusion mechanisms.

8. Summary Table: Representative Multi-Coordinate Clustering Methodologies

Approach/Model	Core Principle	Notable Properties
SSmC (Kernfeld et al., 2014)	Tensor algebra (t-product)	Shift-invariance, module structure, theoretical guarantees
MISC (Wang et al., 2019)	Independent subspaces (ISA)	MDL-based model selection, kernelization
MVMC (Yao et al., 2019)	Shared/individual self-rep. + HSIC	Joint clustering, diversity constraint
DMJC (Lin et al., 2018)	Deep multi-view joint learning	Fusion by soft/hard assignments, end-to-end
Multi-Sub (Yao et al., 2024)	Proxy-based multi-modal latent spaces	User-driven aspect selection, GPT-4/CLIP integration

Each method operationalizes the concept of clustering in multiple coordinate systems using distinct mathematical, statistical, or deep learning constructs, thus contributing to a comprehensive multi-coordinate clustering paradigm suitable for a wide array of contemporary high-dimensional data analysis problems.