Fast Multi-View Anchor Correspondence Clustering

• FMVACC is a multi-view clustering framework that resolves the anchor-unaligned problem by aligning independently learned anchor graphs using feature and structure matching.
• It employs a two-layer matching scheme and columnwise fusion to efficiently consolidate multi-view data, enabling scalable clustering with linear computational cost.
• Empirical evaluations show that FMVACC enhances accuracy and reduces memory usage compared to traditional dense graph methods on large-scale and heterogeneous datasets.

Fast Multi-View Anchor Correspondence Clustering (FMVACC) is a large-scale multi-view clustering framework addressing the anchor-unaligned problem (AUP) via explicit correspondence matching and fusion of anchor graphs learned independently across multiple views. By design, FMVACC achieves both high clustering quality and computational efficiency, scaling linearly with the data size, and has become a canonical approach for multi-view scenarios where data heterogeneity, view-specific feature spaces, or incomplete correspondences hinder traditional graph-based clustering.

1. Formal Problem Definition and Anchor-Unaligned Problem

Given a multi-view dataset with $n$ instances and $v$ views—each view $X_i\in\mathbb R^{n\times d_i}$—the objective is to partition the $n$ samples into $k$ clusters by leveraging all available view information. The FMVACC framework employs anchor-based bipartite graphs, which link instances to a set of $m\ll n$ anchor points via a soft assignment matrix $Z_i\in\mathbb R^{n\times m}$ for each view $i$, where $A_i\in\mathbb R^{m\times d_i}$ are learnable anchor coordinates. This anchor-based representation circumvents the quadratic-to-cubic complexity bottleneck of traditional $n\times n$ adjacency matrices.

The anchor-unaligned problem (AUP) arises because anchor sets $v$0 for different views are constructed independently; their columns represent view-specific cluster prototypes with no guaranteed one-to-one correspondence. This misalignment severely compromises graph fusion and downstream clustering quality unless the columns of the anchor assignment matrices $v$1 are explicitly aligned prior to consensus fusion (Wang et al., 2022).

2. Anchor Graph Construction and Flexible Anchor Generation

Anchor graph construction in FMVACC is performed independently per view by solving the joint optimization: $v$2 subject to $v$3, $v$4, $v$5. The convex QP over $v$6 projects each row to the capped probability simplex, while $v$7 is updated via truncated SVD of $v$8. This step yields discriminative, orthonormal anchors and instance–anchor affinity graphs for each view.

This flexible approach contrasts with earlier methods that selected anchors by k-means on concatenated features, which can result in anchors unrepresentative for some views (Wang et al., 2022, Fang et al., 2022).

3. Anchor Correspondence Alignment: Feature and Structure Matching

FMVACC aligns anchor graphs across views using a two-layered matching scheme.

• First-order (Feature) Correspondence: For each pair $v$9 (using view 1 as template), find a permutation or doubly stochastic matrix $X_i\in\mathbb R^{n\times d_i}$0 that solves:

$X_i\in\mathbb R^{n\times d_i}$1

where $X_i\in\mathbb R^{n\times d_i}$2 is the set of $X_i\in\mathbb R^{n\times d_i}$3 permutation matrices. This is a classic assignment (optimal transport) problem, typically solved by the Hungarian or network-flow algorithms in $X_i\in\mathbb R^{n\times d_i}$4.

• Second-order (Structure) Correspondence: Seek alignment of pairwise anchor similarities by additionally minimizing:

$X_i\in\mathbb R^{n\times d_i}$5

with $X_i\in\mathbb R^{n\times d_i}$6. This quadratic assignment (QAP) captures the global relational structure between anchors.

A unified objective balances both feature and structure matching: $X_i\in\mathbb R^{n\times d_i}$7 weighted by $X_i\in\mathbb R^{n\times d_i}$8. Efficient relaxation to the Birkhoff polytope enables computation via projected fixed-point iteration, with final $X_i\in\mathbb R^{n\times d_i}$9 rounded to a permutation (Wang et al., 2022).

4. Columnwise Fusion and Clustering Objective

After alignment, the anchor graphs $n$0 are fused by columnwise averaging: $n$1 Clustering is performed by extracting the leading $n$2 singular vectors of $n$3 (SVD), followed by $n$4-means on these embeddings. This procedure is efficient, with dominant cost $n$5, where $n$6 is moderate (typically $n$7 suffices empirically) (Wang et al., 2022).

Alternatively, related schemes, such as UDBGL (Fang et al., 2022), impose a discrete Laplacian-rank constraint on the fused bipartite graph to guarantee exactly $n$8 connected components. This allows for direct discrete cluster assignment without a post-hoc $n$9-means step, further reducing runtime and increasing robustness to oversegmentation.

5. Computational Complexity and Algorithmic Steps

For fixed $k$0, FMVACC’s per-iteration cost is $k$1 for anchor construction and $k$2 for alignment (with $k$3 typically small). The memory footprint is $k$4 for $k$5, with additional $k$6 for $k$7. Key steps:

• Anchor graph construction: Each $k$8 is initialized (often via nearest-anchor KNN or random simplex projection).
• Anchor matching: For $k$9, projected fixed-point iterations solve relaxed permutation matching.
• Column-fusion and clustering: Form $m\ll n$0, SVD, and cluster assignment.

All steps admit closed-form or highly efficient subproblems, and the process converges rapidly (∼20 outer iterations). The use of anchors, rather than $m\ll n$1 full graphs, underpins the method’s scalability to $m\ll n$2 (Wang et al., 2022, Fang et al., 2022).

6. Theoretical Connections, Extensions, and Empirical Validation

FMVACC generalizes and bridges several prior paradigms:

• Late Fusion Multi-View Clustering [Liu et al. 2018]: arises as the special case $m\ll n$3 and no structure term ($m\ll n$4); FMVACC admits $m\ll n$5 and imposes richer graph alignment.
• Partial View-Aligned Clustering [Huang et al. 2020]: FMVACC subsumes this by allowing partial one-to-one matching when anchors coincide with instances ($m\ll n$6) and aligning with partial supervision.

Empirical evaluations on benchmarks (3-Sources, UCI-Digit, BDGP, SUNRGBD, MNIST, YTF-10, YTF-20) show that FMVACC achieves or exceeds state-of-the-art clustering accuracy (ACC), normalized mutual information (NMI), and F-score, notably outperforming prior anchor and dense-graph approaches. On simulated data with known ground truth, alignment raises ACC by 10–16% over unaligned baselines and achieves perfect recovery on clusterwise Gaussian mixtures. On large-scale MNIST/YTF-10, FMVACC matches or exceeds lightest baselines while drastically reducing memory compared to dense approaches (Wang et al., 2022).

7. Limitations, Extensions, and Related Methods

FMVACC operates under the assumption that all views are instance-aligned and that view-specific anchors permit columnwise correspondence. Several limitations and extensions are documented:

• Reference-view dependency: Current FMVACC aligns each view to a fixed template; extending to full multiway alignment remains an open challenge.
• Permutation relaxation: Rounding doubly-stochastic alignments to permutations may affect performance for large $m\ll n$7; high structure weight $m\ll n$8 can slow convergence.
• Hyperparameter sensitivity: Optimal anchor count and fusion balance ($m\ll n$9) vary by dataset, suggesting potential benefits from automated selection or adaptive weighting.

Recent works address unaligned or incomplete multi-view scenarios. For instance, SIMVC-SA (Wen et al., 2023) introduces a “soft” orthogonal alignment module, enabling joint anchor learning and cross-view correspondence within a scalable $Z_i\in\mathbb R^{n\times m}$0 joint optimization. PAVuC-ATS (Dong et al., 2024) further generalizes FMVACC by integrating alignment via 2-step Markov chain transitions for the view-unaligned setting, reducing worst-case alignment cost from $Z_i\in\mathbb R^{n\times m}$1 to $Z_i\in\mathbb R^{n\times m}$2 and outperforming FMVACC in scenarios with shuffled correspondences.

Method	Alignment Type	Complexity	Key Feature
FMVACC	Permutation (feature+structure)	$Z_i\in\mathbb R^{n\times m}$3	Explicit matching via optimal transport / QAP
SIMVC-SA	Orthogonal “soft”	$Z_i\in\mathbb R^{n\times m}$4	Simultaneous anchor learning and alignment
PAVuC-ATS	Markov-chain	$Z_i\in\mathbb R^{n\times m}$5	Integrated probabilistic alignment

A plausible implication is that the trend toward integrated, scalable, and probabilistically-aligned anchor learning is leading multi-view clustering methodologies to handle ever more complex, heterogeneous, and unaligned scenarios—without compromising efficiency or clustering accuracy.

---

References:

• “Align then Fusion: Generalized Large-scale Multi-view Clustering with Anchor Matching Correspondences” (Wang et al., 2022)
• “Efficient Multi-view Clustering via Unified and Discrete Bipartite Graph Learning” (Fang et al., 2022)
• “Scalable Incomplete Multi-View Clustering with Structure Alignment” (Wen et al., 2023)
• “Probabilistically Aligned View-unaligned Clustering with Adaptive Template Selection” (Dong et al., 2024)