SuperKMeans: Enhanced K-Means

• SuperKMeans is a framework that extends traditional k-means by incorporating supervision, sparsity constraints, and optimized updates for handling high-dimensional and large-scale data.
• Its methodologies include entropy regularization, fuzzy assignments, and block coordinate descent to seamlessly integrate classification objectives into clustering.
• The approach accelerates vector indexing and similarity search, achieving significant speedups while maintaining high retrieval recall and robust feature selection.

SuperKMeans refers to a set of methodologies and algorithms that enhance, extend, or accelerate the canonical k-means clustering algorithm to address supervised, semi-supervised, high-dimensional, or computationally demanding scenarios. Instantiations of SuperKMeans appear in diverse contexts: as supervised partitioners for classification, constraint-based sparse clusterers for semi-supervised learning, and as highly optimized vector indexing kernels for large-scale embedding collections.

1. Core Motivation and Definitions

SuperKMeans encompasses algorithms that fundamentally depart from vanilla k-means by addressing its limitations in the following domains:

• Supervised and semi-supervised learning: Standard k-means is unsupervised and does not exploit label information. Extensions introduce label-aware loss terms into the clustering objective.
• High-dimensional and sparse data: Feature selection and regularization are critical when many features are irrelevant. SuperKMeans variants incorporate explicit sparsity-inducing constraints or entropy-based regularizers to automatically discard uninformative features.
• Computational efficiency for large-scale vector data: Accelerated implementations target modern hardware by redesigning assignment steps and update rules to improve cache usage, reduce redundant computation, and enable early stopping tied to downstream objectives such as retrieval recall.

This "SuperKMeans" umbrella is not limited to a single algorithm, but refers to several research lines, critically including:

• Supervised Fuzzy Partitioning (SFP): A centroid-based supervised learning method incorporating entropy regularization and arbitrary convex label loss (Ashtari et al., 2018).
• Semi-supervised sparse K-Means: A sparse clustering algorithm leveraging pairwise label constraints and feature weighting (Vouros et al., 2020).
• Super Fast K-means for Indexing: An optimized k-means implementation for high-throughput clustering of embeddings, designed for use in similarity search (Kuffo et al., 20 Mar 2026).
• Voronoi tessellation–based classifiers: Algorithms such as Super-klust use k-means within each class to yield piecewise-linear, prototype-based inference (Zengin et al., 2021).

2. Supervised Fuzzy Partitioning and Connections to k-Means

The SFP framework generalizes k-means to supervised tasks by synthesizing generative and surrogate risk minimization principles. Its model posits a mixture density over both features $x$ and labels $y$:

$$p(x,y) \approx \sum_{j=1}^k \pi_j\, q(x;v_j,w_j)\, f(y;z_j)$$

where $q(x;v_j,w_j)$ is a weighted-Euclidean kernel with feature weights $w_j$ and $f(y;z_j)$ is a label-based exponential-family model or surrogate loss.

The objective function is:

$$\min_{U,V,W,Z} \sum_{i,j}\!u_{ij}\,\|x_i - v_j\|_{w_j}^2 \,+\, \alpha \sum_{i,j}\!u_{ij}\,\ell(y_i,z_j) \,+\, \gamma \sum_{i,j} u_{ij}\ln u_{ij} \,+\, \lambda \sum_{j,l} w_{jl}\ln w_{jl}$$

subject to simplex constraints on both $U$ (membership matrix) and $W$ (feature weights).

The algorithm uses block coordinate descent, alternating between four updates: memberships ($U$), centroids ($y$0), label prototypes ($y$1), and feature weights ($y$2). Fuzzy assignments and entropy-based regularization allow SFP to interpolate between classical k-means, fuzzy c-means, and supervised hard k-means as special cases. The presence of the surrogate loss $y$3 integrates supervision directly, making the approach suitable for classification and regression tasks with arbitrary convex losses (Ashtari et al., 2018).

3. Semi-supervised Sparse SuperKMeans

Another influential branch focuses on semi-supervised clustering with informative feature selection (Vouros et al., 2020). Here, the optimization objective combines:

• Weighted between- and within-cluster scatter (with feature weights $y$4).
• Penalties for violations of must-link and cannot-link constraints (derived from a small labeled subset).
• Explicit $y$5 constraints to induce sparse feature weighting.

The algorithm alternates between assignment (minimizing assignment cost $y$6 considering constraints), centroid updates, and a soft-thresholded feature weight update derived from maximizing the net discriminatory power of features under the current clustering. The closed-form solution to the feature weighting step aligns with established results in sparse clustering literature.

Empirical results show that this method robustly identifies informative features, scales to high-dimensional problems, and significantly outperforms k-means, sparse k-means, and prior constrained clustering methods across synthetic and real-world scenarios. A key insight is that with even 1% of possible constraints, the algorithm successfully prunes uninformative features (weights near zero), unlike diagonal metric learning approaches such as MPCKM (Vouros et al., 2020).

4. SuperKMeans for High-Dimensional Vector Indexing

In large-scale embedding and similarity search, SuperKMeans denotes a highly accelerated variant of Lloyd's algorithm, featuring:

• Random rotation preprocessing to distribute variance uniformly across features, improving the effectiveness of dimension pruning.
• Two-phase assignment: A front subspace partial GEMM (typically on 12.5% of dimensions) quickly prunes centroid candidates by eliminating those that cannot possibly be the nearest.
• Blockwise distance accumulation and early abort in the remaining dimensions minimize memory and compute overhead.
• Early Termination by Recall (ETR): Iterative refinement halts as soon as retrieval recall on held-out queries plateaus, reflecting the primary objective for ANN usage rather than clustering inertia.

This design yields 7×–10× CPU and up to 4× GPU speedup compared to leading libraries (FAISS, Scikit-learn, cuVS) without degrading cluster quality for downstream retrieval tasks, as recall@10 or recall@100 remains within 0.001–0.005 of baseline across multiple datasets and hardware platforms (Kuffo et al., 20 Mar 2026).

Key parameters for controlling computational versus retrieval tradeoffs include the retained front subspace dimension $y$7, block size $y$8 for memory efficiency, and ETR thresholds.

5. Voronoi Tessellation and Piecewise-linear Classification

SuperKMeans-type approaches, specifically Super-klust, exploit k-means clusterings within each class to create prototype-based, piecewise-linear classifiers. Each class is covered by $y$9 clusters, training by running k-means inside each class-specific data subset. Classification for a new point proceeds by assigning it to the label of the nearest prototype (generator), with the decision rule forming a labeled Voronoi tessellation in feature space:

$$p(x,y) \approx \sum_{j=1}^k \pi_j\, q(x;v_j,w_j)\, f(y;z_j)$$0

This construction yields explicit piecewise-constant and piecewise-linear boundaries between regions, determined by the arrangement of the prototypes $$p(x,y) \approx \sum_{j=1}^k \pi_j\, q(x;v_j,w_j)\, f(y;z_j)$$1.

Comparative experiments indicate that Super-klust matches SVMs and k-NN in accuracy (0.909–0.968 on multiple benchmarks), achieves orders-of-magnitude faster inference, and offers direct control over model complexity via $$p(x,y) \approx \sum_{j=1}^k \pi_j\, q(x;v_j,w_j)\, f(y;z_j)$$2 (the number of clusters per class) (Zengin et al., 2021).

6. Comparative Summary of Algorithms and Empirical Findings

Variant/Context	Supervisory Signal	Feature Weighting	Key Empirical Outcomes
SFP / SuperKMeans (Ashtari et al., 2018)	Full supervision (labels)	Per-cluster, entropy reg.	Matches/exceeds SVM, RF; smooth boundaries, fast convergence
Sparse Semi-supervised (Vouros et al., 2020)	Pairs (must-link/cannot-link)	Global, soft thresholding	Outperforms K-means, PCKM, MPCKM for small supervision
Vector Indexing (Kuffo et al., 20 Mar 2026)	Unsupervised (retrieval-oriented)	None	7–10× (CPU) and 4× (GPU) speedup, recall preserved
Super-klust (Zengin et al., 2021)	Full supervision (per class)	None	Matches SVM, k-NN on real data; explicit decision regions

Empirical results consistently show that SuperKMeans instantiations retain or improve upon the accuracy of standard k-means and common baselines. They deliver additional benefits in terms of feature interpretability, computational performance, or flexibility for supervised and semi-supervised tasks, depending on the implementation.

7. Connections, Scope, and Future Directions

SuperKMeans represents a principled evolution of centroid-based methods towards more flexible, efficient, and interpretable clustering and classification in the presence of supervision, constraints, or scaling demands. Core themes that unify these developments include blockwise or closed-form optimization, entropy- or sparsity-inducing regularization, and adaptation to task-specific objectives (e.g., retrieval recall).

Distinctions exist between variants: SFP generalizes k-means to the supervised setting with soft assignments and feature weighting, semi-supervised sparse SuperKMeans targets constrained clusterings with interpretable sparsity, and vector indexing SuperKMeans focuses on throughput and recall optimization for ANN pipelines. Prototype-based classifiers such as Super-klust leverage k-means in a multi-class setting to generate explicit region-based decision boundaries.

A plausible implication is that as the scale of unlabeled data, the importance of computational resource constraints, and the need for feature selection in high-dimensional settings continue to increase, further research will converge on hybrid approaches that unify these principles—aligning clustering, classification, and retrieval tasks under a scalable, supervised, and interpretable SuperKMeans framework.