Iterative Clustering Techniques

• Iterative clustering techniques are methods that repeatedly update assignments, centroids, or latent structures to progressively improve cluster quality and robustness.
• They integrate strategies such as seeded initialization, consensus refinement, and semi-supervised feedback to effectively manage high-dimensional, noisy, or sparsely labeled datasets.
• These techniques find practical applications in areas like image segmentation, text analysis, and anomaly detection, offering convergence guarantees and enhanced performance over single-pass methods.

An iterative clustering technique is any clustering method that applies its core cluster-assignment, feature-transformation, or cluster-structure update in a multi-pass, cyclic, or recursive fashion—rather than in a single batch pass. These algorithms leverage the repeated alternation of assignment, parameter updating, or consensus refinement to improve cluster quality, estimate the number of clusters, handle noise/anomalies, adapt to interactive user feedback, or fuse outputs from multiple partitioning schemes. Iterative refinement is critical in high-dimensional, noisy, or weakly supervised settings, and underlies much of the recent progress in scalable, robust, semi-supervised, and interpretable clustering.

1. Mathematical Principles and Prototypical Algorithms

Central to iterative clustering is the alternation of a clustering-related operation (assignment, merging, centroid update, or consensus formation) with an update to an associated representation or a latent structure. Typical frameworks include:

• Iterative mean/median shift: Classical $K$-means or $\ell_1$-median clustering alternately updates cluster assignments and centroids until convergence. Extensions such as "Probabilistic $\ell_1$ Method for Clustering High Dimensional Data" explicitly use weighted coordinate-wise medians and probabilistic assignment updates in each iteration, with convergence guarantees and monotonic descent of objective (Asamov et al., 2015).
• Seeded iterative clustering: In "Seeded iterative clustering for histology region identification," cluster centroids are initialized using sparse user seeds, and assignments are repeatedly refined by recalculating centroids from the current seed set and reassigning points, yielding rapid convergence to a partition with strong agreement to the seed-induced labels (Chelebian et al., 2022).
• Iterative consensus refinement: Consensus clustering frameworks such as ICC construct a similarity matrix from multiple clustering runs, then iteratively refine the matrix by reclustering and thresholding to accentuate block-diagonality and stabilize the Perron cluster structure in its associated Markov chain, ultimately revealing the natural number of clusters (Race et al., 2014, Race et al., 2014).
• Cluster-wise PCA alternation: Iterative Complement Clustering PCA models data as a sum of global (homogeneity) and group-specific (sub-homogeneity) low-rank factors, alternating between cluster-wise PCA, global PCA over cluster-level PCs, and leave-one-out reassignment to globally minimize reconstruction error (Bi et al., 2022).
• Interactive/robust iterative assignment: Some frameworks, such as iterative classification for short-text clustering, repeatedly remove outliers, retrain discriminative models (e.g., multinomial logistic regression), and reassign fringe points based on updated models, with each loop designed to boost cluster tightness and reduce noise sensitivity (Rakib et al., 2020).
• Iterative spectral methods: In directed-graph clustering, spectral embeddings of dynamically-updated Hermitian matrices are repeatedly recomputed and $k$-means assignments updated—a cycling procedure shown to outperform single-pass spectral methods on pattern-driven digraphs (Martin et al., 29 Jan 2025).
• Iterative feature selection + clustering: High-dimensional sparse data clustering alternates between variable selection (thresholding a discriminating direction) and SDP-relaxed K-means, converging to exact recovery even in the $p \gg n$ regime (Mun et al., 26 May 2025).

2. Seeded, Semi-supervised, and Interactive Iteration

Iterative clustering enables efficient incorporation of external (often sparse) supervision or user feedback:

• Seeded techniques: In histopathology, patch-level annotations are extremely expensive, motivating approaches where sparse user seeds initialize centroids. Each iteration restricts updates to the current "positive" region and re-centers clusters using only embedded patches indexed by current seeds (Chelebian et al., 2022).
• Semi-supervised anomaly-based expansion: Iterative, semi-supervised anomaly-driven clustering alternates between expanding clusters from seeds (based on cluster-wise robustness principles and anomaly detection), ejecting new anomalies, and absorbing non-anomalous fringe points—resulting in parameter-free, robust clustering particularly well suited for exploratory or incomplete labeling settings (Mohammad, 2023).
• Human-in-the-loop Bayesian refinement: Tinder clustering implements a feedback-driven Bayesian prior elicitation model, where a user may accept/reject clusters, and the posterior is iteratively modified to enforce deviation from previously seen solutions. A stochastic block-coordinate descent alternately updates assignments and estimates parameters (Srivastava et al., 2016).
• Iterative outlier detection and reclassification: Short-text clustering is improved by repeated outlier removal (Isolation Forest), classification modeling (logistic regression), and re-assignment of former outliers, typically leading to statistically significant improvements over non-iterative clustering baselines (Rakib et al., 2020).

3. Consensus and Multi-algorithm Iterative Refinement

Iterative consensus techniques explicitly exploit the diversity of multiple clustering algorithms or runs across different hyperparameter settings:

• Consensus similarity matrix refinement: ICC builds an $n \times n$ matrix by counting co-clustering occurrences across ensemble runs, then aggressively denoises and block-diagonalizes the matrix via thresholding and reclustering, focusing on the emergence of spectral gaps encoding true cluster number (Race et al., 2014, Race et al., 2014). The block-diagonalization is amplified at each iteration, enabling robust detection of true clusters even in high-noise or high-dimensional contexts.
• Integrated spectral alternation for digraphs: The iterative spectral algorithm for clustering directed graphs repeatedly updates both the cluster assignment and the underlying Hermitian matrix embedding edge directionality, allowing it to search a broader solution space (meta-graphs) than any single spectral relaxation (Martin et al., 29 Jan 2025).

4. Iterative Clustering for Domain-Specific Applications

Specialized domains leverage iterative clustering to address high dimensionality, noise, label scarcity, or interpretability:

• Self-supervised and contrastive iterative loops: Cluster-aware Iterative Contrastive Learning for scRNA-seq data alternates between transformer-based embedding learning, K-means re-centering, pseudo-label updating via Student's $t$ kernel, and cluster-aware contrastive loss refinement, producing representations with state-of-the-art performance on 25 benchmarks (Jiang et al., 2023).
• Unsupervised object localization: The iterative spectral clustering approach for object localization repeatedly shrinks bounding box candidate sets by spectral bipartitioning and scores, then groups remaining proposals post-hoc, yielding unsupervised localization on par with weakly supervised object detection pipelines (Vora et al., 2017).
• Iterative superpixel algorithms: Both SLIC and its noise-robust extension Fuzzy SLIC apply spatially-constrained, local fuzzy C-means or distance-weighted assignments in an iterative manner, enabling precise, robust, and fast superpixel decomposition for image segmentation (Margapuri et al., 2022, Wu et al., 2018).
• Outlier-robust solutions for big data: Iterative subsampling solution path clustering achieves scalability by alternating between clustering a small random subsample with concave penalization and sequentially reassigning the remainder by likelihood ratio, allowing for rapid clustering and denoising of datasets with $n=100$K+ (Marchetti et al., 2014).

5. Convergence, Complexity, and Robustness Properties

• Convergence: Most iterative techniques guarantee monotonic objective improvement or label stabilization. For example, probabilistic $\ell_1$ clustering is descent-monotone, ICC converges once the Perron cluster size stabilizes, while iterative semi-supervised methods halt after no new labels are changed (Asamov et al., 2015, Race et al., 2014, Mohammad, 2023).
• Computational complexity: While each iteration may be expensive (e.g., full matrix SVD in consensus methods, SDP solve in sparse clustering), the number of necessary iterations is typically small (3–10 for ICC, tens for CPM-based semi-supervised schemes). Implementation-wise, local update steps (coordinate-wise medians, K-means, partial eigen-decompositions) are easily parallelized.
• Robustness: Iterative alternation—with block diagonalization, outlier ejection, re-centering by robust statistics (such as medians), or consensus aggregation—yields substantial gains in noise tolerance, recovery of small or weak clusters, anomaly detection, and resistance to the "masking" by dominant clusters, all quantitatively confirmed by simulation and real-data experiments (Race et al., 2014, Mohammad, 2023, Tepper et al., 2011).

6. Illustrative Empirical Results

Method / Domain	Data Scenario	Metric/Performance
Seeded Iterative Clustering (Histopathology)	DigestPath tumor segmentation	F₁-score (SimCLR features): 0.74 ± 0.23
Iterative Consensus Clustering (ICC)	Text, Newsgroups (NG6, n=1,800)	Consensus achieves accuracy 99%, best single-algorithm 98% (Race et al., 2014)
Fuzzy SLIC (Superpixels)	BSD500 images, noisy (σ=0.2)	5–15% higher CDBR than SLIC, 10–30% lower under-segmentation error
Iterative Spectral (Digraph Clustering)	Food webs, neural connectome	δ clustering score improved ~15–53% over baseline (Martin et al., 29 Jan 2025)
Iterative SDP K-means (Sparse Clustering)	$p=5,000$, $n=200$	Recovers true clusters, competitors fail as $p \gg n$ (Mun et al., 26 May 2025)
Iterative Subsampling SPC (Gene expression)	$n=100,000$, $p=100$	10–100$\times$ speedup over batch SPC, ARI within 2–3% (Marchetti et al., 2014)

In all cases, iterative refinement yields both performance and robustness improvements unattainable by single-pass clustering, especially in the presence of outliers, high dimensionality, or ambiguous cluster structure.

7. Limitations, Parameterization, and Theoretical Considerations

• Parameter sensitivity: Most iterative techniques require choice of seed set size, stopping thresholds, or consensus parameters, but many demonstrate insensitivity or supply data-driven heuristics (e.g., ARI threshold in CPCA (Bi et al., 2022), consensus intolerance τ in ICC).
• Non-convexity and local minima: Solutions may depend on initialization (especially in spectral or EM-based frameworks), but multiple restarts, consensus iteration, or robustness-based pruning can mitigate the risk of poor solutions.
• Scalability: Despite per-iteration cost, various incremental or subsampling approaches (e.g., iterative subsampling SPC (Marchetti et al., 2014), iterative greedy assignment (Alfares et al., 2019)) scale to 100,000+ objects with either negligible loss in quality or even improved anomaly detection.
• Theoretical guarantees: Select methods (e.g., sparse SDP K-means (Mun et al., 26 May 2025), CPCA (Bi et al., 2022), probabilistic $\ell_1$ clustering (Asamov et al., 2015)) provide finite-sample or asymptotic optimality, including minimax separation bounds for exact recovery. Others afford only empirical or monotonic descent guarantees due to the inherent nonconvexity.

Iterative clustering techniques constitute a central paradigm for modern unsupervised learning, enabling principled, robust, and scalable discovery of latent structure across heterogeneous, noisy, or large-scale data. The algorithmic core—alternating assignment, updating, and consensus—has been instantiated in a wide range of both classical and domain-specific clustering methodologies.