CvxCluster: Convex Clustering & Resource Allocation

• CvxCluster is a family of convex clustering methods that use fusion penalties to derive continuous fusion paths and guarantee a unique global minimizer.
• It employs advanced algorithmic strategies such as ADMM and CARP to efficiently compute regularization paths over large-scale and high-dimensional datasets.
• A distinct application adapts these convex techniques for cluster resource allocation, achieving significant speed-ups and near-optimal performance in large computing systems.

CvxCluster denotes a family of methods centered on convex clustering: given data points $x_1,\dots,x_n\in\mathbb R^p$, one introduces one centroid $u_i$ per observation and estimates all centroids by solving a convex optimization problem that balances fidelity to the data against a fusion penalty on pairwise centroid differences. As the regularization parameter increases from $0$ to $\infty$, the solution traces a continuous fusion path from $n$ singleton clusters to one cluster, yielding a hierarchical structure without spurious local minima and with a unique global minimizer (Chi et al., 11 Jul 2025, Weylandt et al., 2019). In the cited literature, the same name is also used for a distinct two-stage convex method for cluster resource allocation in computing systems (Nnorom et al., 2 May 2026).

1. Canonical optimization formulations

The basic convex clustering problem introduces a matrix $U\in\mathbb R^{n\times p}$ with rows $u_1,\dots,u_n$ and solves

$$\min_{U\in\mathbb R^{n\times p}} \frac12\sum_{i=1}^n \|x_i-u_i\|_2^2 +\gamma\sum_{i<j} w_{ij}\,\|u_i-u_j\|_q,$$

where $\gamma\ge 0$ is the regularization parameter, $w_{ij}\ge 0$ are nonnegative fusion weights, and $u_i$0 is typically $u_i$1, though $u_i$2 and $u_i$3 also appear in the literature. The first term is a least-squares fidelity term, and the second is a group-lasso–style fusion penalty that shrinks differences $u_i$4 toward zero (Chi et al., 11 Jul 2025).

This formulation has a simple path interpretation. When $u_i$5, each centroid equals its observation, so the solution consists of $u_i$6 clusters. As $u_i$7, all centroids collapse to the grand mean $u_i$8. Intermediate values produce partial fusions, and the ordered sequence of fusions defines a fusion path that can be viewed as a continuous dendrogram (Chi et al., 11 Jul 2025).

A closely related weighted total-variation formulation replaces the $u_i$9 fusion norm by an $0$0 norm,

$0$1

In Xu et al., the weights are chosen by a Gaussian kernel,

$0$2

and one often sparsifies $0$3 by keeping only the $0$4 nearest neighbors of each point. This weighted-TV model was proposed to revisit total variation based convex clustering and to sharpen exact clustering guarantees for general data (Xu et al., 2018).

Across formulations, the weights $0$5 determine which pairs are encouraged to fuse first. Common choices in the cited literature include uniform weights, inverse-distance weights, Gaussian-kernel weights, $0$6-nearest-neighbor graphs, minimum spanning trees, MST + $0$7-NN graphs, and disjoint MSTs. Sparse graphs with $0$8 both reduce computational cost and often improve clustering quality (Chi et al., 11 Jul 2025).

2. Algorithmic frameworks and path computation

Fast solvers usually begin from the equality-constrained splitting

$0$9

This reformulation isolates the nonsmooth fusion penalty in $\infty$0, whose proximal map is group soft-thresholding. It underlies ADMM, AMA, semismooth-Newton ALM, warm-started path algorithms, compression schemes, adaptive sieving, and stochastic or incremental methods (Chi et al., 11 Jul 2025).

In ADMM-based solvers, the $\infty$1-step solves a linear system involving the graph Laplacian, the $\infty$2-step applies proximal shrinkage edgewise, and the dual step updates multipliers. In AMA, the dual update becomes a projection onto norm balls and can be accelerated to $\infty$3. SSNAL uses an augmented-Lagrangian method with a semismooth Newton inner solve and empirically achieves superlinear convergence in a few outer steps (Chi et al., 11 Jul 2025).

Weylandt, Nagorski, and Allen introduced Algorithmic Regularization for convex clustering and, from it, the CARP algorithm, “Convex Clustering via Algorithmic Regularization Paths.” The central modification is to replace “iterate ADMM to convergence at each $\infty$4” by exactly one ADMM update, then increase the regularization parameter by a multiplicative factor $\infty$5, then take one more update. With $\infty$6, $\infty$7, $\infty$8, and $\infty$9 the edge-difference operator, CARP repeatedly performs one $n$0-update, one proximal $n$1-update, one dual $n$2-update, and then sets $n$3. Back-tracking can be added when more than one fusion event is detected at once, so that fusion ordering is exact or near-exact for dendrogram construction (Weylandt et al., 2019).

The computational motivation is explicit. Classical convex clustering path computation requires many inner iterations at each regularization value. CARP trades exactness at any single $n$4 for an ultra-fine regularization grid. On genomics and text data sets, CARP was reported to deliver over a 100-fold speed-up over existing methods while attaining a finer approximation grid than standard methods; in the detailed benchmarks, standard ADMM/AMA on 100 $n$5-grid points took 6–20 hours, whereas CARP with $n$6 finished in under a minute and recovered approximately 95% of fusion events, and CARP with $n$7 finished in about 5 minutes and recovered 100% of tree events (Weylandt et al., 2019).

A distinct algorithmic line is the proximal-distance method. There, one introduces auxiliary variables $n$8, replaces the hard linear constraint by a distance-to-constraint penalty, majorizes the resulting objective, and alternates explicit centroid updates with shrinkage updates for $n$9. The associated software separates parameters, accommodates missing data, supports prior information on relationships, and is implemented in C++ with OpenCL on ATI and NVIDIA GPUs (Chen et al., 2014).

3. Theoretical properties and recovery guarantees

The foundational theoretical properties of convex clustering are consequences of convexity and strong convexity. The objective is free of spurious local minima, the minimizer $U\in\mathbb R^{n\times p}$0 is unique, and the solution is jointly continuous in the data $U\in\mathbb R^{n\times p}$1, the tuning parameter $U\in\mathbb R^{n\times p}$2, and the weights $U\in\mathbb R^{n\times p}$3. The survey literature also states a $U\in\mathbb R^{n\times p}$4-Lipschitz stability property in the data: $U\in\mathbb R^{n\times p}$5 These properties distinguish convex clustering from nonconvex procedures such as $U\in\mathbb R^{n\times p}$6-means (Chi et al., 11 Jul 2025).

Perfect recovery results are formulated as separation conditions. For uniform weights, the survey gives

$U\in\mathbb R^{n\times p}$7

and states that exact recovery occurs whenever $U\in\mathbb R^{n\times p}$8. With adaptive weights, corresponding bounds $U\in\mathbb R^{n\times p}$9 can permit recovery even when the uniform-weight interval is empty (Chi et al., 11 Jul 2025).

Xu et al. sharpened this line of analysis for weighted total variation based convex clustering. In the deterministic setting, if two clusters satisfy

$u_1,\dots,u_n$0

then one can choose $u_1,\dots,u_n$1 and $u_1,\dots,u_n$2 sufficiently large so that the unique minimizer exactly recovers cluster membership. For general $u_1,\dots,u_n$3 clusters, if all cluster means are distinct in each coordinate and

$u_1,\dots,u_n$4

then there again exist $u_1,\dots,u_n$5 so that the minimizer exactly recovers all membership. Probabilistic exact-recovery statements are also given for the stochastic-ball model and a Gaussian-mixture model (Xu et al., 2018).

The theoretical literature also records important limitations. Without strong separation, convex clustering can fail; the survey cites the case of two “nearby balls” failing to split correctly. It also states that the method is not outlier-robust under an $u_1,\dots,u_n$6 loss unless the loss is replaced by a resistant alternative such as Huber. These negative results temper overly broad claims about universal recovery (Chi et al., 11 Jul 2025).

For CARP specifically, the main informal result is a pathwise approximation theorem: as the step-size $u_1,\dots,u_n$7 and the initial $u_1,\dots,u_n$8, the CARP path $u_1,\dots,u_n$9 converges in Hausdorff distance to the exact primal and dual solution path, under easily checked non-data-dependent assumptions consisting of strong convexity of the loss, Lipschitz continuity of the solution path, and linear convergence of the underlying ADMM iteration (Weylandt et al., 2019).

4. High-dimensional variants, tuning, and implementation

Convex clustering in high dimensions can be distorted by uninformative features. Sparse Convex Clustering addresses this by augmenting the convex clustering objective with an adaptive group-lasso penalty on feature columns: $$\min_{U\in\mathbb R^{n\times p}} \frac12\sum_{i=1}^n \|x_i-u_i\|_2^2 +\gamma\sum_{i<j} w_{ij}\,\|u_i-u_j\|_q,$$0 When $$\min_{U\in\mathbb R^{n\times p}} \frac12\sum_{i=1}^n \|x_i-u_i\|_2^2 +\gamma\sum_{i<j} w_{ij}\,\|u_i-u_j\|_q,$$1, one recovers convex clustering; when $$\min_{U\in\mathbb R^{n\times p}} \frac12\sum_{i=1}^n \|x_i-u_i\|_2^2 +\gamma\sum_{i<j} w_{ij}\,\|u_i-u_j\|_q,$$2, the group penalty enforces global sparsity of entire features. Wang et al. developed both sparse ADMM and sparse AMA solvers, an unbiased estimator of degrees of freedom, and a stability-based tuning criterion over a two-dimensional $$\min_{U\in\mathbb R^{n\times p}} \frac12\sum_{i=1}^n \|x_i-u_i\|_2^2 +\gamma\sum_{i<j} w_{ij}\,\|u_i-u_j\|_q,$$3 grid (Wang et al., 2016).

The stability-based criterion repeatedly bootstraps the rows of $$\min_{U\in\mathbb R^{n\times p}} \frac12\sum_{i=1}^n \|x_i-u_i\|_2^2 +\gamma\sum_{i<j} w_{ij}\,\|u_i-u_j\|_q,$$4, fits SCC on paired bootstrap samples, extracts cluster assignments by fusing identical rows, computes a Rand index, and chooses $$\min_{U\in\mathbb R^{n\times p}} \frac12\sum_{i=1}^n \|x_i-u_i\|_2^2 +\gamma\sum_{i<j} w_{ij}\,\|u_i-u_j\|_q,$$5 to maximize average agreement. Theoretical results in the same work include finite-sample prediction error bounds under sub-Gaussian noise and variable-selection consistency for zero-columns in high-dimensional regimes (Wang et al., 2016).

Practical tuning guidance is consistent across the cited papers. Weight matrices are usually sparsified by $$\min_{U\in\mathbb R^{n\times p}} \frac12\sum_{i=1}^n \|x_i-u_i\|_2^2 +\gamma\sum_{i<j} w_{ij}\,\|u_i-u_j\|_q,$$6-nearest neighbors; Gaussian-kernel weights or inverse-distance weights are common; warm-starts along a regularization path are standard; and $$\min_{U\in\mathbb R^{n\times p}} \frac12\sum_{i=1}^n \|x_i-u_i\|_2^2 +\gamma\sum_{i<j} w_{ij}\,\|u_i-u_j\|_q,$$7 may be selected by hold-out validation, stability selection, or information criteria using an unbiased degrees-of-freedom estimate (Chi et al., 11 Jul 2025, Wang et al., 2016).

Implementation ecosystems reflect two complementary emphases. The clustRviz package implements CARP and dynamic visualization; its carp() interface returns $$\min_{U\in\mathbb R^{n\times p}} \frac12\sum_{i=1}^n \|x_i-u_i\|_2^2 +\gamma\sum_{i<j} w_{ij}\,\|u_i-u_j\|_q,$$8-, $$\min_{U\in\mathbb R^{n\times p}} \frac12\sum_{i=1}^n \|x_i-u_i\|_2^2 +\gamma\sum_{i<j} w_{ij}\,\|u_i-u_j\|_q,$$9-, and $\gamma\ge 0$0-paths, regularization values, and dendrogram objects, and an analogous cbass() function supports convex bi-clustering with row and column dendrograms and a clustered heatmap (Weylandt et al., 2019). The scvxclustr package implements Sparse Convex Clustering (Wang et al., 2016). Earlier CvxCluster software built around the proximal-distance algorithm additionally accommodates missing data by majorizing unobserved entries with the current iterate and supports composite distances that combine multiple side-information metrics in the weight definition (Chen et al., 2014).

5. Visualization, applications, and empirical behavior

One of the stated impediments to wider adoption of convex clustering was its lack of compelling visualizations. CARP addresses this by producing a fine solution grid that supports a convex clustering-based dendrogram and dynamic path-wise visualization. In the clustRviz framework, one can draw dendrograms with merge heights equal to $\gamma\ge 0$1, animate the projected trajectories of $\gamma\ge 0$2 on the first two or three principal components, and link the evolving dendrogram to a scatter-path display. These dynamic visualizations are implemented using htmlwidgets and D3 (Weylandt et al., 2019).

The broader applications catalog is extensive. The survey literature lists biclustering and tensor clustering, clustered regression or network lasso, sparse convex clustering, integration with PCA, LLE, and CCA, spatial neuroscience, genomics, market segmentation, image segmentation, and network smoothing plus clustering (Chi et al., 11 Jul 2025). In bioinformatics, hierarchical clustering has been visually attractive but prone to false inferences in the presence of outliers and noise; convex clustering was presented as preserving the visual appeal of hierarchical clustering while ameliorating those problems (Chen et al., 2014).

Illustrative biological studies in the proximal-distance CvxCluster software include breast-cancer transcriptome clustering, global human genetic diversity using HGDP, and European population structure using POPRES. In the breast-cancer data, the path recovered two stable cluster centers separating ER-positive and ER-negative tumors, with a few outliers appearing naturally as singleton centroids before coalescing. In the population-genetics examples, composite genetic plus geographic distances yielded continental and intracontinental structures that merged in geographic or migration-consistent order (Chen et al., 2014).

Empirical comparisons in the weighted-TV line show strong performance on synthetic and real datasets. Xu et al. report that CvxCluster slightly beats Lloyd’s and $\gamma\ge 0$3-means++ on Gaussian mixtures and is markedly superior on a non-convex “two-circles” example, where CvxCluster achieves Rand $\gamma\ge 0$4 with zero standard deviation while Lloyd’s and $\gamma\ge 0$5-means++ remain near $\gamma\ge 0$6. On Iris, AR-face, Libras movement, and Leaf, CvxCluster is described as either top-ranked or tied against Lloyd’s $\gamma\ge 0$7-means, $\gamma\ge 0$8-means++, single-linkage, and average-linkage (Xu et al., 2018).

In high-dimensional settings with many irrelevant features, the sparse variant is materially different from ordinary convex clustering. In the reported simulations, standard convex clustering broke down when $\gamma\ge 0$9, whereas SCC maintained Rand approximately $w_{ij}\ge 0$0–$w_{ij}\ge 0$1 with low false-positive and false-negative rates for feature selection; on LIBRAS hand movement, SCC with 13 selected features achieved Rand approximately $w_{ij}\ge 0$2, compared with approximately $w_{ij}\ge 0$3 for convex clustering using all 90 features and approximately $w_{ij}\ge 0$4 for $w_{ij}\ge 0$5-means (Wang et al., 2016).

6. CvxCluster as a two-stage cluster resource allocator

In a separate systems literature, CvxCluster names a method for cluster resource allocation rather than data clustering. The starting point is a mixed-integer placement model with binary decision variables $w_{ij}\ge 0$6 indicating whether task $w_{ij}\ge 0$7 is placed on server $w_{ij}\ge 0$8, together with assignment, capacity, and anti-affinity constraints. Because the binary formulation is NP-hard, the method relaxes $w_{ij}\ge 0$9 to $u_i$00, producing a linear program labeled $u_i$01 (Nnorom et al., 2 May 2026).

The dual variables of this LP have a direct interpretation. Multipliers $u_i$02 are the marginal shadow prices of resource $u_i$03 on server $u_i$04, and $u_i$05 are the prices of using one more anti-affinity slot of group $u_i$06 on server $u_i$07. The algorithm then proceeds in two stages. Stage 1 aggregates servers by shape, solves the convex relaxation, and extracts shape-level prices. Stage 2 computes a net utility

$u_i$08

sorts tasks by best net utility, and greedily attempts feasible placements on servers of the preferred shape, falling back to next-best shapes if necessary (Nnorom et al., 2 May 2026).

The reported performance is large-scale. On Azure traces, CvxCluster scales to 100,480 servers under proportional workload growth and sustains arrival rates up to $u_i$09 the baseline trace. It runs $u_i$10 to $u_i$11 faster than a state-of-the-art MIP solver while remaining within $u_i$12 of the optimal objective, and it supports complex constraints such as job anti-affinity, machine types, and GPU servers (Nnorom et al., 2 May 2026). Detailed experiments also report shape-based speedups of $u_i$13–$u_i$14, a throughput of $u_i$15 tasks/s on a $u_i$16-server cluster replaying a $u_i$17-day Azure trace, a $u_i$18 end-to-end throughput advantage over DCM with OR-Tools CP-SAT on a real $u_i$19-node AWS cluster in KWOK mode, and exact objective matches to MIP on all solvable GPU-aware benchmarks while achieving up to $u_i$20 speedup (Nnorom et al., 2 May 2026).

This systems usage shares with convex clustering the central design principle of replacing a hard discrete search with a convex relaxation and then exploiting the resulting structure. A plausible implication is that the name “CvxCluster” now refers not only to convex clustering in statistics, but also to a broader convex-optimization strategy for large, structured cluster-level decision problems.