Recursive HDBSCAN: Multiscale Density Clustering

• Recursive HDBSCAN is a framework that integrates inherent recursive clustering with exploratory re-runs to reveal subclusters and recover noise pockets.
• It employs mutual reachability measures and a condensed tree to build a multiscale hierarchy, ensuring stability through persistence metrics.
• Recent approaches like PLSCAN formalize recursive clustering by summarizing leaf cluster persistence across scales, enhancing robustness and parameter sensitivity.

Recursive HDBSCAN denotes a family of practices built around HDBSCAN* rather than a single standardized algorithm. In one sense, recursion is internal to HDBSCAN*: single-linkage on mutual reachability produces a recursive hierarchy, condensation recursively prunes small branches, and flat cluster selection is solved on the condensed tree. In another sense, recursion is exploratory: analysts re-run HDBSCAN* on previously found clusters to reveal substructure, re-run it on points labeled as noise to recover small dense pockets, or sweep the minimum cluster size across a range to identify useful resolutions. Recent work on "Persistent Multiscale Density-based Clustering" formalizes this latter usage through PLSCAN, which constructs one multiscale description of all HDBSCAN* leaf clusters across all minimum cluster sizes and uses persistence to identify the stable clusterings that manual recursive sweeps are intended to find (Bot et al., 18 Dec 2025).

1. Meanings of recursion in HDBSCAN

In the HDBSCAN* literature, recursion has two complementary meanings. The first is structural. HDBSCAN* builds a hierarchy of clusters and subclusters by performing single-linkage clustering in the mutual reachability metric, condenses that hierarchy with a minimum cluster size constraint, and then selects a disjoint set of stable clusters from the condensed tree. The second is operational. Practitioners often recursively re-run HDBSCAN* on subtrees or subclusters to refine local structure or to retune parameters to local density scales (McInnes et al., 2017).

A closely related description appears in work on FISHDBC, where recursion is both algorithmic and exploratory. Algorithmically, the hierarchy itself is a tree of clusters and subclusters obtained by progressively relaxing or tightening a density threshold. Exploratorily, one traverses that tree recursively to refine some branches, coarsen others, or select a final clustering by choosing nodes that best summarize the data (Dell'Amico, 2019). This usage aligns with common practice in exploratory data analysis, where one alternates between coarse density structure and local drill-down.

This dual meaning is important because it separates HDBSCAN* as a hierarchical density estimator from "recursive HDBSCAN" as a workflow pattern. The former is intrinsic to the algorithm; the latter is a response to the fact that a single flat extraction is often not sufficient when clusters exist at multiple density scales.

2. Canonical HDBSCAN* machinery

HDBSCAN* begins by replacing the base metric with mutual reachability distances. In the notation used by PLSCAN, for points $x_i$ and $x_j$,

$$d_{mut}(x_i, x_j) = \max\{ \kappa(x_i), \kappa(x_j), d(x_i, x_j) \},$$

where $\kappa(x_i)$ is the core distance of $x_i$, defined as the distance to its $k$-th nearest neighbor. Larger $k$ smooths the density estimates by inflating local distances in high-density regions and suppressing small, sharp peaks (Bot et al., 18 Dec 2025). In the original HDBSCAN* notation, for parameter $m=\text{min\_samples}$,

$$d_{mreach}(x,y) = \max\{ d_{core}(x), d_{core}(y), d(x,y) \},$$

with $d_{core}(x)$ the distance from $x_j$0 to its $x_j$1-th nearest neighbor (McInnes et al., 2017).

Distances are then converted to density scale by

$x_j$2

or equivalently $x_j$3 in the standard HDBSCAN* parameterization. Single-linkage on the mutual reachability metric yields a hierarchy equivalent to DBSCAN* across all $x_j$4, and condensation with minimum cluster size $x_j$5 prunes away branches that never reach size $x_j$6. Leaf clusters correspond to local density maxima, while increasing $x_j$7 smooths the modeled density profile by removing small peaks (Bot et al., 18 Dec 2025).

Flat cluster extraction is then performed on the condensed tree. HDBSCAN* supports leaf selection and stability-based selection. In the stability formulation, for a condensed-tree branch $x_j$8,

$x_j$9

and cluster selection solves

$$d_{mut}(x_i, x_j) = \max\{ \kappa(x_i), \kappa(x_j), d(x_i, x_j) \},$$0

An equivalent formulation expresses stability via relative excess of mass (McInnes et al., 2017). This machinery explains why recursive re-running is appealing: flat selection necessarily compresses a rich multiscale tree into one layer.

3. Manual recursion as a multiscale workflow

The practical patterns usually called recursive HDBSCAN are explicit in recent descriptions of analyst behavior. Three common workflows recur: sweeping the minimum cluster size to see which settings reveal stable clusters, re-clustering large clusters to find substructure at smaller minimum cluster size, and re-clustering points labeled as noise to salvage small dense pockets (Bot et al., 18 Dec 2025). These are not separate algorithms; they are repeated uses of the same HDBSCAN* pipeline at different scales or on restricted subsets.

The same literature also records the trade-offs of recursive refinement. Re-running HDBSCAN* within a selected branch may provide finer local adaptation of parameters and better separation of subtle subclusters, but it risks losing global density context, introducing parameter drift across recursion levels, duplicating noise points across levels, and overfitting to local fluctuations (McInnes et al., 2017). If core distances and mutual reachability are recomputed on a subset, the resulting density geometry is not the same as the geometry inherited from the full dataset.

This suggests that recursive HDBSCAN is best understood as an attempt to recover a multiscale description that the standard flat output suppresses. The central question is therefore not whether recursion is possible, but how to obtain the full range of relevant scales without repeated ad hoc reruns.

4. Persistent leaves and PLSCAN

PLSCAN, "Persistent Leaves Spatial Clustering for Applications with Noise," was introduced precisely to support this multiscale, recursive intent in a principled, efficient, and comprehensive way (Bot et al., 18 Dec 2025). Its central object is the persistent leaf: a leaf cluster that remains a leaf across a range of minimum cluster sizes. Rather than repeatedly re-running HDBSCAN* by hand, PLSCAN constructs a leaf tree whose segments encode when a leaf appears and disappears as $$d_{mut}(x_i, x_j) = \max\{ \kappa(x_i), \kappa(x_j), d(x_i, x_j) \},$$1 varies:

$$d_{mut}(x_i, x_j) = \max\{ \kappa(x_i), \kappa(x_j), d(x_i, x_j) \},$$2

Here $$d_{mut}(x_i, x_j) = \max\{ \kappa(x_i), \kappa(x_j), d(x_i, x_j) \},$$3 and $$d_{mut}(x_i, x_j) = \max\{ \kappa(x_i), \kappa(x_j), d(x_i, x_j) \},$$4 are the minimum cluster sizes at which the segment becomes and ceases to be a leaf cluster, and $$d_{mut}(x_i, x_j) = \max\{ \kappa(x_i), \kappa(x_j), d(x_i, x_j) \},$$5 are the mutual reachability distance bounds for that segment’s existence in the condensed tree.

The leaf tree is built from the condensed tree by two rules. For each merge, the death size of both children is set to the smaller sibling size,

$$d_{mut}(x_i, x_j) = \max\{ \kappa(x_i), \kappa(x_j), d(x_i, x_j) \},$$6

and the parent’s birth is delayed until all downstream leaves have disappeared,

$$d_{mut}(x_i, x_j) = \max\{ \kappa(x_i), \kappa(x_j), d(x_i, x_j) \},$$7

Any segment with $$d_{mut}(x_i, x_j) = \max\{ \kappa(x_i), \kappa(x_j), d(x_i, x_j) \},$$8 is a leaf cluster for $$d_{mut}(x_i, x_j) = \max\{ \kappa(x_i), \kappa(x_j), d(x_i, x_j) \},$$9 (Bot et al., 18 Dec 2025).

Stability is then quantified by size persistence,

$\kappa(x_i)$0

and total persistence at a fixed $\kappa(x_i)$1 is obtained by summing $\kappa(x_i)$2 over all leaves existing at that $\kappa(x_i)$3. Because the leaf set changes only at discrete $\kappa(x_i)$4 and $\kappa(x_i)$5 values, PLSCAN evaluates persistence only at those candidate values and identifies local maxima of the trace. These maxima are the stable clusterings at different scales. The method also supports distance persistence, density persistence using a compressed density

$\kappa(x_i)$6

and bi-persistences that integrate size with distance or density lifetimes (Bot et al., 18 Dec 2025).

The algorithm mirrors HDBSCAN* up to the condensed tree and then diverges. It computes a mutual-reachability MST and single-linkage tree, builds a condensed tree

$\kappa(x_i)$7

constructs the leaf tree in increasing merge-distance order, computes the persistence trace over all candidate $\kappa(x_i)$8, selects one or more cluster layers at local maxima, and derives labels and membership probabilities by assigning data points to their selected-leaf parent. A single condensed tree built with a sufficiently small initial $\kappa(x_i)$9 already contains every merge event that will appear at higher $x_i$0; increasing $x_i$1 only prunes branches, without changing merge distances or sizes. This is why PLSCAN can recover all topology-changing minimum-cluster-size thresholds without brute-force re-running across a dense grid (Bot et al., 18 Dec 2025).

5. Topological formulation and related recursive frameworks

PLSCAN gives recursive HDBSCAN a formal topological interpretation. Let $x_i$2 be the set of leaves after pruning at scale $x_i$3, and define

$x_i$4

This function satisfies the ultrametric inequality

$x_i$5

Because leaves can die before parents appear, the construction augments the space with special elements $x_i$6 capturing noise events, with

$x_i$7

and

$x_i$8

Zero-dimensional persistent homology on $x_i$9 recovers the same birth and death events as the leaf tree, so the computational procedure is equivalent to persistent homology on a novel metric space (Bot et al., 18 Dec 2025).

Other work addresses adjacent forms of recursive or multiscale HDBSCAN. "Efficient Computation of Multiple Density-Based Clustering Hierarchies" computes all HDBSCAN* hierarchies for a range of $k$0 values by building a single sparse graph that contains all required MST edges for every $k$1 in a chosen interval, making multi-$k$2 exploration practical rather than recomputing a fresh hierarchy for each value (Neto et al., 2017). This differs from PLSCAN, which fixes $k$3 and varies only $k$4.

FISHDBC approximates HDBSCAN* with an incremental strategy based on an approximate MST maintained from HNSW candidate edges. It supports interactive, recursive exploration of clusters in arbitrary or non-metric distances and under streaming updates, but the hierarchy remains an approximation unless all distances are computed (Dell'Amico, 2019). FLASC extends HDBSCAN* in a different direction: it is a post-processing refinement that detects branches within selected clusters by a second filtration over a centrality graph. The method is explicitly not recursive HDBSCAN, because it does not re-run density clustering on subclusters; it performs branch extraction inside an existing HDBSCAN* solution (Bot et al., 2023). Dynamic summarization work based on Bubble-tree supports a multi-resolution recursive strategy by clustering summaries at coarse levels and refining only selected subtrees, while geometric reconstruction in Euclidean space yields an exact divide-and-conquer HDBSCAN* pipeline in which large interiors are discarded and only boundary-plus-noise interfaces are propagated between stages (Abduaziz et al., 2024, Garcia-Pulido et al., 2022).

6. Empirical behavior, limitations, and practical interpretation

Empirically, PLSCAN was compared to HDBSCAN* on several real-world datasets. At $k$5, the reported average ARI values were 0.58 for HDBSCAN* EOM, 0.16 for HDBSCAN* leaf, 0.72 for PLSCAN size, 0.51 for PLSCAN size–d, 0.60 for PLSCAN size–$k$6, 0.46 for PLSCAN d, and 0.55 for PLSCAN $k$7. The datasets included CIFAR-10, Fashion-MNIST, MNIST, 20 Newsgroups, Iris, Audioset (music), multiple UCI datasets, and biological datasets, with UMAP used to reduce dimensionality for fairness and runtime (Bot et al., 18 Dec 2025).

Sensitivity to the neighbor parameter also differs. HDBSCAN* performances dip substantially at small $k$8, especially with leaf selection, while EOM is better but still more sensitive. PLSCAN with size, size–d, and size–$k$9 persistences exhibits markedly lower sensitivity across $k$0, and using the best top-5 layers per $k$1 further improves robustness. Under the reported sensitivity protocol, scores $k$2 are classified as insensitive, and PLSCAN’s distributions concentrate in the insensitive regime more than HDBSCAN* (Bot et al., 18 Dec 2025). Runtime is dominated by nearest-neighbor queries and mutual-reachability MST construction, as in HDBSCAN*. In low dimensions, PLSCAN is competitive with, and often faster than, scikit-learn’s k-Means with 100 clusters; in higher dimensions, runtime scales similarly to HDBSCAN*, with both dominated by nearest-neighbor and MST steps (Bot et al., 18 Dec 2025).

Several limitations constrain recursive use and its replacements. PLSCAN’s default selection uses a single $k$3 cut across the leaf tree, so datasets with very unequal cluster sizes or strongly two-sided hierarchies can bias total persistence toward large structures and under-select smaller meaningful clusters. Tree-based nearest-neighbor acceleration degrades in high dimensions, so scaling resembles HDBSCAN* in those regimes. Mutual reachability flattening remains sensitive to $k$4 and to the base metric, and certain datasets with extreme size differences may still benefit from EOM in HDBSCAN* rather than size persistence (Bot et al., 18 Dec 2025). More generally, if recursive refinement is performed by physically re-clustering subsets, the density context changes with the subset, which is not equivalent to slicing a global hierarchy or global multiscale filtration (McInnes et al., 2017).

For practice, the explicit guidance associated with PLSCAN is to start with $k$5 and the size persistence measure, inspect the persistence trace, select the highest peak as the default solution, and examine other local maxima as alternative stable resolutions. This replaces manual recursion over clusters or noise with inspection of multiple cluster layers derived from one computation. A plausible implication is that "recursive HDBSCAN" is most coherent when treated as multiscale model selection over a single hierarchical object rather than as repeated, manually retuned reruns of the same algorithm (Bot et al., 18 Dec 2025).