Multi-Coordinates Clustering

• Multi-coordinates clustering is an unsupervised technique that integrates heterogeneous data views to reveal both unified and alternative cluster structures in complex datasets.
• It employs optimization-based, Bayesian, and deep learning methods to fuse complementary information while mitigating noise across different observation spaces.
• Empirical studies demonstrate its improved performance in diverse applications such as image recognition, text mining, and biomedical data, with robust scalability and convergence guarantees.

Multi-coordinates clustering, also referred to as multi-view clustering or multi-modal clustering, denotes a comprehensive class of unsupervised learning methods where data points are observed in multiple coordinate spaces (views), and the task is to integrate these heterogeneous representations for more accurate or diverse clustering. Each view may represent different sensor modalities, feature descriptors, or variable subspaces. The aim is either to infer a consensus clustering, aligned across all views, or to discover multiple, potentially orthogonal clusterings reflecting the data's underlying multi-faceted structure. Recent research presents model-driven, optimization-based, algebraic, and deep learning frameworks for multi-coordinates clustering in high-dimensional and large-scale settings.

1. Problem Formulation and Conceptual Foundations

Let $n$ denote the number of samples and $V$ the number of available views. For view $v=1,\ldots,V$, the data is given as $X^{(v)}\in\mathbb{R}^{d_v\times n}$, with $d_v$ the dimensionality of view $v$. Multi-coordinates clustering addresses settings where clustering in one view alone fails to capture the underlying data structure due to noise, bias, or incompleteness; instead, a shared or integrated clustering is sought, possibly leveraging complementary information or diverse data mechanisms.

Two primary conceptual paradigms exist:

• Consensus/Integrated Clustering: Derive a single clustering partition $Y\in\{0,1\}^{k\times n}$ reflecting shared structure present across all views, with $k$ the number of clusters. The challenge is to aggregate information such that noise or idiosyncratic structure in a single view does not dominate the result (Kong et al., 28 Jan 2024).
• Multiple/Alternative Clusterings: Identify several distinct clusterings, each corresponding to different “facets” of the data, recognizing that in high-dimensional, multi-view contexts, multiple meaningful partitions may coexist (Duan, 2019, Yao et al., 2019, Wei et al., 2019).

The term "coordinates" is contextually overloaded: it can refer to variable/feature directions, to coordinate systems defined by feature subspaces, or to partition spaces (cluster indicator matrices).

2. Model Classes and Representative Methods

Modern approaches to multi-coordinates clustering include, but are not limited to:

2.1. Dual-space Anchor-based Fusion

The "Dual-space Co-training Large-scale Multi-view Clustering" (DSCMC) approach (Kong et al., 28 Jan 2024) introduces projections in both original and latent shared spaces. It jointly learns:

• Orthogonal view-specific projections $P^{(v)}$ (original space) and feature transforms $W^{(v)}$ (shared latent space), both mapping onto $k$-dimensional anchors.
• A discriminative anchor graph via an orthogonal matrix $A$ and a non-negative matrix $Z$, encoding cluster assignments and sample similarity.
• An explicit loss coupling original-space "complementarity" ($\sum_{v=1}^V\|X^{(v)}-P^{(v)}AZ\|_F^2$) and latent-space "consistency" ($\sum_{v=1}^V\|W^{(v)}X^{(v)}-AZ\|_F^2$), with sparsity and regularization terms.

Among the key design decisions is element-wise integration (all views are weighted equally, $\alpha_v\equiv 1$), sidestepping the instability associated with adaptive weighting in ambiguous scenarios.

2.2. Bayesian and Nonparametric Multi-view Co-clustering

Tokuda et al. (Tokuda et al., 2015) propose a nonparametric Bayesian multiple co-clustering framework, where the number and structure of views, row clusters, and feature clusters are inferred automatically via stick-breaking Dirichlet process priors. Each view is modeled as a block-structured mixture of distributions (e.g., Gaussians, Poisson, multinomial) across observed data types. Variational Bayes with mean field approximation enables tractable inference, and unused views or clusters are pruned automatically in the posterior.

2.3. Latent Simplex Position and Random Partition Models

The Latent Simplex Position (LSP) model (Duan, 2019) constructs, for each view $m$, an $n\times K$ simplex coordinate matrix $Z^{(m)}$ where each row encodes assignment uncertainty for the $K$ clusters. Pairwise co-assignment matrices $P^{(m)} = Z^{(m)}[Z^{(m)}]^T$ are optimized to fit observed similarity matrices well in the sense of pairwise Kullback–Leibler divergence. The model supports fewer than $V$ distinct coordinate templates, encouraging sharing across views, with rigorous PAC-Bayes generalization guarantees.

2.4. Deep Multi-view and Matrix Factorization Approaches

Deep joint multi-view clustering frameworks, such as DMJC (Lin et al., 2018), employ parallel deep encoder networks for each view, followed by view-fusion via soft assignments or auxiliary targets and optimization via KL divergence objectives. Multiple distinct clusterings can be discovered via multi-layer factorization (DMClusts (Wei et al., 2019)), where each layer yields a non-negative representation whose clustering assignments are encouraged to be mutually diverse via a redundancy penalty.

2.5. Partition-alignment and Free Module Algebraic Methods

Partition-space fusion (mPAC (Kang et al., 2019)) aligns the continuous partition indicators from each view via Procrustes rotation before fusing into a discrete consensus, with automatic view-weighting. Sparse Submodule Clustering (SSmC (Kernfeld et al., 2014)) generalizes subspace clustering to the t-product setting, representing samples as tubal vectors in a free module, enforcing affinity via tubal-convolution self-representation.

3. Optimization Objectives and Algorithms

The methods in this domain typically construct and optimize composite objectives reflecting complementarity and consensus across views, as well as view-specific structure. Regularization terms are used to promote sparsity, diversity, or smoothness. The following summarizes typical components and algorithmic principles:

Model	Objective Composition	Main Optimization Procedure
DSCMC (Kong et al., 28 Jan 2024)	Original-space + latent-space losses, anchor-graph factorization, element-wise fusion	Block coordinate descent with SVD, closed-form, and QP updates for P, A, W, Z
Tokuda et al. (Tokuda et al., 2015)	Nonparametric blockwise likelihood + Dirichlet-process priors	Variational Bayes EM, closed-form updates
LSP (Duan, 2019)	KL between observed similarity and low-rank co-assignment; simplex constraints	EM with gradient descent; simplex projection
DMJC (Lin et al., 2018)	KL divergence between fused and view-specific cluster assignments	End-to-end SGD/backpropagation
DMClusts (Wei et al., 2019)	Multi-layer reconstruction + diversity penalty	Alternating optimization with closed-form/multiplicative updates
mPAC (Kang et al., 2019)	Reconstruction + partition alignment + auto-weighted fusion	Alternating minimization: closed-form, SVD, Procrustes

These approaches frequently employ alternating minimization, with blockwise update rules tailored to each parameter group.

4. Theoretical Properties and Guarantees

The multi-coordinates clustering literature has produced theoretical analyses along several axes:

• Identifiability & Exactness: For SSmC (Kernfeld et al., 2014), exact cluster recovery is established under conditions on submodule independence and tubal angle coherence, leveraging algebraic properties of the t-product.
• Generalization and Risk Control: LSP (Duan, 2019) formalizes the link between the KL-based objective and PAC-Bayes risk bounds for multiple random partition recovery, guaranteeing control of generalization gap for the unknown partitions.
• Convergence: Blockwise coordinate descent procedures, Procrustes alignment, and convex subproblems ensure monotonic decrease of objectives and convergence to stationary points in both optimization-based and Bayesian frameworks. mPAC (Kang et al., 2019) provides detailed per-block complexity analysis and practical convergence rates.

5. Experimental Results, Benchmarks, and Applications

Multi-coordinates clustering methods have demonstrated strong empirical performance on a variety of challenging benchmarks, including multi-modal image datasets (Caltech101, Yale, MNIST, VGGFace), text corpora (WebKB, BBCSport, Reuters), multi-omics and biomedical data, and large-scale web collections (Kong et al., 28 Jan 2024, Tokuda et al., 2015, Wei et al., 2019).

Key quantitative findings include:

• DSCMC (Kong et al., 28 Jan 2024) achieves highest or near-highest normalized mutual information (NMI), accuracy (ACC), F-score, and adjusted Rand index (ARI) on all nine reported datasets, with particularly large gains (10% absolute ACC) on Caltech101-7 and robust scaling to 120k samples.
• Nonparametric multiple co-clustering (Tokuda et al., 2015) recovers both view and cluster structure (ARI > 0.7) in synthetic and real settings, robust to 20% missing data.
• Deep matrix factorization (DMClusts (Wei et al., 2019)) outperforms baselines on Silhouette Coefficient, Dunn Index, and exhibits high diversity (low NMI, low Jaccard Coefficient) in produced clusterings.
• mPAC (Kang et al., 2019) improves F-score and NMI substantially over traditional kernel, graph, and NMF-based multi-view methods across all reported datasets.

Application domains include high-dimensional biomedical data (gene expression, clinical phenotypes), image retrieval and recognition, text mining, and semantic search over vector embeddings, particularly where isotropy and invariance to coordinate axes are important (Sadikov et al., 2016, Kernfeld et al., 2014).

6. Practical Considerations and Current Limitations

• Complexity and Scalability: Recent methods achieve nearly linear per-iteration complexity in the number of samples for moderate $k$ and number of views. DSCMC (Kong et al., 28 Jan 2024) is a prototypical example. Greedy or brute-force split heuristics in isotropic clustering are practical due to bounded node degree (Sadikov et al., 2016).
• Anchor Selection and Model Hyperparameters: Some approaches (e.g., DSCMC) require preselection of the number of clusters or anchors. Automatic inference is possible in Bayesian methods, but at increased computational cost (Tokuda et al., 2015).
• View Integration: Element-wise weighting often offers more robust integration than learned adaptive view weights, especially when views vary significantly in relevance or noise (Kong et al., 28 Jan 2024, Kang et al., 2019).
• Diversity and Multiple Clustering: Redundancy penalties (e.g., HSIC (Yao et al., 2019), trace-regularized overlap (Wei et al., 2019)) are effective for enforcing diversity among clusterings, each reflecting different data structure.

7. Synthesis and Directions

Multi-coordinates clustering is an established field at the intersection of unsupervised learning, multi-view modeling, and matrix/tensor factorization, advancing both in algorithmic design and theoretical guarantees. Recent progress confirms that leveraging the geometry, uncertainty, and diversity inherent in multiple views leads to quantifiably improved cluster recovery, scalability, and interpretability, especially in complex, real-world settings where signal is fragmented across observation spaces. Outstanding directions include fully unsupervised selection of anchor quantities, principled model selection under weak supervision, unification with manifold regularization for graph-structured data, and further exploration of algebraic and deep nonlinear embedding frameworks (Kong et al., 28 Jan 2024, Wei et al., 2019, Kang et al., 2019).