Convex Clustering: A Global Optimization Framework

• Convex clustering is a method that reformulates clustering as a single convex optimization problem using fusion penalties to merge centroids, bridging k-means and hierarchical methods.
• It employs a continuous regularization parameter that creates a clusterpath, smoothly transitioning from singleton clusters to a single fused cluster.
• Advanced algorithms like ADMM, MM, and proximal gradient approaches enable scalable solutions for high-dimensional and large-scale applications in genomics, image analysis, and more.

Convex clustering is a clustering framework that formulates clustering as a single convex optimization problem rather than as a combinatorial search over partitions. In its standard form, each observation is assigned its own centroid variable, and a fusion penalty encourages centroids to coalesce as a regularization parameter increases. The result is a globally optimal solution for each regularization level, together with a clusterpath that interpolates between $n$ singleton clusters and a single fused cluster, thereby combining features of $k$-means and hierarchical agglomerative clustering (Zhang et al., 31 Mar 2025, Chen et al., 2014).

1. Standard formulation and basic mechanism

Given observations $\mathbf{x}_1,\dots,\mathbf{x}_n \in \mathbb{R}^p$, a canonical convex clustering problem is

$$\hat U_\lambda = \arg\min_{U \in \mathbb{R}^{n\times p}} \left\{ \frac{1}{2}\|X-U\|_F^2 + \lambda \sum_{1\le i<j\le n} w_{ij}\,\|U_{i\cdot}-U_{j\cdot}\|_q \right\},$$

where $U_{i\cdot}$ is the centroid attached to observation $i$, $\lambda>0$ is a fusion parameter, $w_{ij}\ge 0$ are graph weights, and $\|\cdot\|_q$ is typically an $\ell_1$, $k$0, or $k$1 norm. Clusters are defined by equality of centroids: $k$2 indicates that observations $k$3 and $k$4 belong to the same cluster. At $k$5, one recovers $k$6, so each sample is its own cluster; for sufficiently large $k$7, all centroids fuse into a single cluster (Zhang et al., 31 Mar 2025, Weylandt et al., 2019).

The squared data-fit term keeps each centroid close to its observation, while the fusion penalty shrinks centroid differences toward zero. Because the objective is convex, the optimization has a global solution; with the usual squared loss it is strongly convex, and uniqueness follows in the standard Euclidean formulation. This is the principal formal contrast with $k$8-means, whose objective is nonconvex and requires fixing the number of clusters $k$9, and with hierarchical agglomerative clustering, whose merges are greedy and are not obtained as minimizers of a single global objective (Chen et al., 2014).

Weights $\mathbf{x}_1,\dots,\mathbf{x}_n \in \mathbb{R}^p$0 define the interaction graph. Dense complete-graph formulations emphasize global coupling; sparse $\mathbf{x}_1,\dots,\mathbf{x}_n \in \mathbb{R}^p$1-nearest-neighbor or tree-structured graphs emphasize locality and computational tractability. In many implementations, Gaussian-kernel weights are used, often after graph sparsification, so that nearby points receive large penalties for remaining separate while distant points are coupled weakly (Xu et al., 2018, Touw et al., 2022).

2. Geometry, recovery, and theoretical guarantees

Two complementary theoretical strands characterize convex clustering. One studies exact recovery in weighted total-variation $\mathbf{x}_1,\dots,\mathbf{x}_n \in \mathbb{R}^p$2 models. A representative formulation is

$\mathbf{x}_1,\dots,\mathbf{x}_n \in \mathbb{R}^p$3

In this setting, exact clustering means that $\mathbf{x}_1,\dots,\mathbf{x}_n \in \mathbb{R}^p$4 if and only if $\mathbf{x}_1,\dots,\mathbf{x}_n \in \mathbb{R}^p$5 and $\mathbf{x}_1,\dots,\mathbf{x}_n \in \mathbb{R}^p$6 belong to the same ground-truth cluster. Deterministically, if

$\mathbf{x}_1,\dots,\mathbf{x}_n \in \mathbb{R}^p$7

then there exist $\mathbf{x}_1,\dots,\mathbf{x}_n \in \mathbb{R}^p$8 such that the minimizer has the exact clustering property; for the stochastic ball model, exact clustering holds with high probability when $\mathbf{x}_1,\dots,\mathbf{x}_n \in \mathbb{R}^p$9; for Gaussian mixtures, a corresponding high-probability statement is given under a covariance-dependent separation condition, with probability at least $$\hat U_\lambda = \arg\min_{U \in \mathbb{R}^{n\times p}} \left\{ \frac{1}{2}\|X-U\|_F^2 + \lambda \sum_{1\le i<j\le n} w_{ij}\,\|U_{i\cdot}-U_{j\cdot}\|_q \right\},$$0 (Xu et al., 2018).

A separate line of analysis studies the geometry of solutions of the unweighted sum-of-norms problem. There the conclusion is restrictive and precise: convex clustering can only learn convex clusters. If $$\hat U_\lambda = \arg\min_{U \in \mathbb{R}^{n\times p}} \left\{ \frac{1}{2}\|X-U\|_F^2 + \lambda \sum_{1\le i<j\le n} w_{ij}\,\|U_{i\cdot}-U_{j\cdot}\|_q \right\},$$1 denotes the points assigned to a prototype $$\hat U_\lambda = \arg\min_{U \in \mathbb{R}^{n\times p}} \left\{ \frac{1}{2}\|X-U\|_F^2 + \lambda \sum_{1\le i<j\le n} w_{ij}\,\|U_{i\cdot}-U_{j\cdot}\|_q \right\},$$2, and $$\hat U_\lambda = \arg\min_{U \in \mathbb{R}^{n\times p}} \left\{ \frac{1}{2}\|X-U\|_F^2 + \lambda \sum_{1\le i<j\le n} w_{ij}\,\|U_{i\cdot}-U_{j\cdot}\|_q \right\},$$3 is the convex hull of $$\hat U_\lambda = \arg\min_{U \in \mathbb{R}^{n\times p}} \left\{ \frac{1}{2}\|X-U\|_F^2 + \lambda \sum_{1\le i<j\le n} w_{ij}\,\|U_{i\cdot}-U_{j\cdot}\|_q \right\},$$4, then $$\hat U_\lambda = \arg\min_{U \in \mathbb{R}^{n\times p}} \left\{ \frac{1}{2}\|X-U\|_F^2 + \lambda \sum_{1\le i<j\le n} w_{ij}\,\|U_{i\cdot}-U_{j\cdot}\|_q \right\},$$5 for distinct clusters. Moreover, each cluster lies in a Euclidean ball

$$\hat U_\lambda = \arg\min_{U \in \mathbb{R}^{n\times p}} \left\{ \frac{1}{2}\|X-U\|_F^2 + \lambda \sum_{1\le i<j\le n} w_{ij}\,\|U_{i\cdot}-U_{j\cdot}\|_q \right\},$$6

and for $$\hat U_\lambda = \arg\min_{U \in \mathbb{R}^{n\times p}} \left\{ \frac{1}{2}\|X-U\|_F^2 + \lambda \sum_{1\le i<j\le n} w_{ij}\,\|U_{i\cdot}-U_{j\cdot}\|_q \right\},$$7 the centers satisfy

$$\hat U_\lambda = \arg\min_{U \in \mathbb{R}^{n\times p}} \left\{ \frac{1}{2}\|X-U\|_F^2 + \lambda \sum_{1\le i<j\le n} w_{ij}\,\|U_{i\cdot}-U_{j\cdot}\|_q \right\},$$8

Thus cluster centers are empirical means, cluster radii scale linearly with both $$\hat U_\lambda = \arg\min_{U \in \mathbb{R}^{n\times p}} \left\{ \frac{1}{2}\|X-U\|_F^2 + \lambda \sum_{1\le i<j\le n} w_{ij}\,\|U_{i\cdot}-U_{j\cdot}\|_q \right\},$$9 and cluster size, and distinct clusters are separated by a nontrivial gap (Nguyen et al., 2021).

These results have several implications. They explain why convex clustering is effective on well-separated convex structure, but also why it is intrinsically ill-suited to interlocking, curved, or otherwise non-convex clusters. They also imply that the regularization parameter has a direct geometric meaning: it governs not only fusion pressure but also admissible within-cluster spread and between-cluster separation (Nguyen et al., 2021).

3. Clusterpaths, dendrograms, and hierarchical structure

The solution path

$U_{i\cdot}$0

is central to convex clustering. Each $U_{i\cdot}$1 yields a partition through equality of centroids, and sweeping $U_{i\cdot}$2 from $U_{i\cdot}$3 to large values produces a multiscale family of partitions. This motivates the view of convex clustering as a continuous analogue of hierarchical clustering, and it underlies path visualizations, dendrogram-like summaries, and model-selection procedures based on the regularization path (Weylandt et al., 2019, Chen et al., 2014).

Path computation is, however, a nontrivial problem. For the preferred $U_{i\cdot}$4 fusion penalty, the path is not piecewise linear, so LARS-type exact path algorithms do not apply. A naive strategy solves the convex program on a grid of $U_{i\cdot}$5-values, but coarse grids can miss fusion events and therefore distort the implied hierarchy (Weylandt et al., 2019).

A further complication is structural rather than numerical: cluster splitting can occur in general convex clustering. If two centroids are equal at one regularization level, they need not remain equal at a larger one, and sparse weight graphs can prevent the path from terminating in a single final cluster. These facts mean that an unconstrained convex-clustering path is not automatically a proper dendrogram (Touw et al., 2022).

Several recent methods address this directly. Algorithmic Regularization and CARP compute fine approximations to the regularization path by taking one optimization step per regularization update; the resulting approximate path converges globally to the exact one in the Hausdorff sense under non-data-dependent assumptions, and in applications CARP yields over a 100-fold speed-up while enabling convex clustering-based dendrograms and dynamic path-wise visualizations (Weylandt et al., 2019). CCMM and TGCC take a different route: they explicitly fuse clusters during optimization and then solve reduced problems over the fused representatives. In TGCC, the constraint that previously fused pairs remain fused guarantees no cluster splitting and yields a monotone hierarchy, while cluster fusion along tree edges preserves the tree structure needed by the solver (Zhang et al., 31 Mar 2025). CCMM similarly integrates irreversible fusions into the optimization path, thereby enforcing a hierarchical structure by construction (Touw et al., 2022).

4. Algorithms and computational regimes

The algorithmic literature on convex clustering is unusually broad because the same objective can be solved under very different graph and norm assumptions. Early and general approaches relied on generic convex solvers, splitting methods such as ADMM and AMA, semismooth Newton methods such as SSNAL, and MM-based procedures (Zhang et al., 31 Mar 2025, Touw et al., 2022). Proximal-distance methods reparameterize centroid differences with auxiliary variables, solve a sequence of majorized subproblems, accommodate missing data, and support prior information through weighted graphs; implementations on ATI and nVidia GPUs were reported for high-dimensional biological applications (Chen et al., 2014).

For large sparse graphs, first-order smoothing has proved effective. The smoothing proximal gradient algorithm Sproga replaces the non-smooth fusion term by a Nesterov-smoothed approximation and applies an accelerated proximal-gradient method. In the reported experiments it is faster than ADMM- or AMA-based convex clustering algorithms by one to two orders of magnitude, and its memory requirement is at least one order of magnitude smaller than that of ADMM and AMA (Zhou et al., 2020).

Majorization-minimization has also been used to make convex clustering scale to very large $U_{i\cdot}$6. CCMM combines cluster fusions with a diagonal majorization of the fused objective, so that each iteration has cost effectively linear in the number of nonzero graph edges. In the reported large-scale experiment, CCMM solved convex clustering problems with over one million objects in seven-dimensional space in 51 seconds on average on a current desktop computer (Touw et al., 2022).

Tree-structured $U_{i\cdot}$7 convex clustering opens an even more specialized computational regime. For $U_{i\cdot}$8, the loss separates across coordinates,

$U_{i\cdot}$9

so each feature becomes a one-dimensional fused lasso on the graph defined by $i$0. When the weights are supported on a tree, this subproblem can be solved exactly by dynamic programming; TGCC couples that solver with tree-based cluster fusion and can construct a complete clusterpath for $i$1 points in $i$2 within 15 seconds on a standard laptop without parallel or distributed computing frameworks (Zhang et al., 31 Mar 2025).

5. Structured, supervised, and alternative extensions

Convex clustering has been extended far beyond unsupervised Euclidean sample clustering. One alternative formulation views clustering through optimal transport: the empirical distribution is approximated by a sparse-support distribution in Wasserstein distance, and the support-cardinality penalty is relaxed through a group-lasso penalty on the columns of the transport plan. In that setting, the sum-of-norms relaxation is the tightest convex lower approximation of the cardinality penalty on the feasible probability-constrained set (Carli et al., 2013).

For graph-structured data, where the input is a similarity matrix rather than explicit vectors, convex clustering can be formulated over a doubly-stochastic matrix $i$3 whose columns represent centroids as convex combinations of nodes. The objective combines a kernelized reconstruction term with a mixed total-variation penalty on $i$4, and a proximal dual algorithm based on FISTA is used to recover the regularization path (Donnat et al., 2019).

Supervision can be incorporated directly. Supervised Convex Clustering introduces a joint convex fusion penalty on augmented centroids $i$5, where $i$6 encodes feature structure and $i$7 encodes a supervising auxiliary variable through a GLM loss. This framework extends to continuous, binary, count, categorical, and survival outcomes, allows covariate adjustment, and has biclustering variants that cluster both subjects and features (Wang et al., 2020). A related construction clusters covariates rather than samples by penalizing differences between columns $i$8 of a multiclass logistic-regression coefficient matrix, weighted by an a-priori covariate-similarity graph; a specialized ADMM solver was proposed for this feature-clustering problem (Andrade et al., 2019).

The same sum-of-norms principle also appears in personalized federated learning. There the variables are user-specific models $i$9, the per-user losses are arbitrary convex functions $\lambda>0$0, and the objective

$\lambda>0$1

induces automatic model clustering without requiring the number of clusters in advance. Analytical lower and upper bounds on $\lambda>0$2 describe regimes where within-cluster fusion occurs without collapse across clusters, and a PDMM algorithm realizes the method in a server–client architecture (Armacki et al., 2022). Tree-guided convex clustering adds biclustering and sparse convex clustering through a Dykstra-like proximal algorithm, reusing the same tree-guided row and column solvers (Zhang et al., 31 Mar 2025).

6. Robustness, applications, and limitations

The classical least-squares formulation is not robust. In resistant convex clustering, replacing the squared loss by the Huber loss yields

$\lambda>0$3

and the paper proves a sharp breakdown-point contrast: least-squares convex clustering has breakdown point $\lambda>0$4 for any $\lambda>0$5, whereas the Huber-plus-fusion estimator has breakdown point between $\lambda>0$6 and $\lambda>0$7 under

$\lambda>0$8

This shows that the fusion penalty can enhance resistance when paired with a robust loss, but not under ordinary least squares (Sun et al., 2019). More recent work such as COMET combines convex-clustering structure with Median-of-Means aggregation and clipped fusion; that line improves robustness but explicitly departs from strict convexity, so it is best viewed as a robust descendant rather than a standard convex clustering method (De et al., 12 Nov 2025).

Applications are correspondingly diverse. Reported case studies include world-wide microsatellite data, European SNP data, breast cancer gene-expression profiles, genomics and text-analysis datasets, connectomics, Alzheimer’s disease genomics, and single-cell RNA-seq (Chen et al., 2014, Weylandt et al., 2019, Donnat et al., 2019, Wang et al., 2020, Zhou et al., 2020). Across these domains, convex clustering is valued for global optimality, multiscale structure, and the possibility of integrating domain graphs or supervision into a single objective.

Its limitations are equally structural. The strongest is geometric: in the unweighted sum-of-norms formulation, convex clustering can only learn convex clusters with disjoint bounding balls and significant gaps, so genuinely non-convex cluster shapes lie outside its solution class (Nguyen et al., 2021). The second is path-related: unconstrained paths can split clusters or terminate in multiple connected components if the weight graph is disconnected, which undermines a literal dendrogram interpretation (Touw et al., 2022). The third is computational trade-off: methods such as TGCC achieve exceptional speed by imposing tree-structured weights, but they do not directly handle arbitrary $\lambda>0$9-NN or fully connected graphs, and in higher dimensions the minimum-spanning-tree construction itself can become the bottleneck (Zhang et al., 31 Mar 2025).

Taken together, these results place convex clustering in a precise position within clustering theory. It is neither merely a convex surrogate for $w_{ij}\ge 0$0-means nor simply a continuous hierarchical method. It is a family of globally solvable fusion models whose behavior is controlled by the interaction of loss geometry, graph structure, norm choice, and path constraints. Its most successful uses arise when those ingredients are aligned with the underlying scientific structure.