---
title: HDBSCAN-MC Algorithm
url: https://www.emergentmind.com/topics/hdbscan-mc-algorithm
type: topic
---

# HDBSCAN-MC Algorithm

Searching arXiv for the cited papers to ground the article in current records.
HDBSCAN-MC denotes a family of HDBSCAN\*-derived procedures rather than a single universally standardized algorithmic object. In the arXiv literature, the label is used most explicitly for two distinct purposes: an efficient method for computing multiple HDBSCAN\* hierarchies over a range of `mpts` values by reusing one reduced graph [1709.04545], and a Monte Carlo–propagated HDBSCAN\* pipeline for uncertainty-aware cluster membership estimation in Gaia DR3 Young Stellar Object catalogs [2507.09721]. Related work also describes accelerated, thresholded, and geometrically reconstructed HDBSCAN\* variants that are sometimes interpreted as belonging to the same broader methodological space, although several of those papers do not use the exact name “HDBSCAN-MC” [1705.07321] [1911.02282] [2203.08020].

## 1. Core HDBSCAN\* machinery

HDBSCAN\* is a hierarchical density-based clustering method organized around the parameter `mpts` or `minPts`, interpreted as a minimum-points or smoothing factor. It can be viewed as a natural extension of DBSCAN, a hierarchical version of DBSCAN\*, and an MST-based or robust single-linkage method. Its central construction is the mutual reachability graph, whose edge weights are defined from core distances and ordinary metric distances [1705.07321] [1709.04545].

For a point \(p\), the core distance with respect to `mpts` is
\[
\cd(p)=d(p, mpts\operatorname{-NN}(p)).
\]
For two points \(p,q\), the mutual reachability distance is
\[
\mrd(p,q)=\max\{\cd(p),\cd(q),d(p,q)\}.
\]
This distance inflates separations through sparse regions by incorporating local density via the core distances. In HDBSCAN\*, the hierarchy is recovered from the minimum spanning tree of the complete graph weighted by mutual reachability distance. For a fixed threshold \(\varepsilon\), thresholding the mutual reachability graph yields the same connected components as DBSCAN\* at that scale, and the MST suffices because the hierarchy can be reconstructed by removing edges in descending order [1705.07321] [2203.08020].

A further HDBSCAN\* layer is the condensed tree. The hierarchy is pruned using a minimum cluster size \(m\): clusters smaller than \(m\) are removed, one surviving child can be treated as the continuation of its parent, and true splits occur when more than one child has size at least \(m\). Flat cluster extraction is then commonly posed as a stability-maximization problem over the condensed tree, subject to non-overlap constraints [1705.07321] [1911.02282].

## 2. HDBSCAN-MC as multiple-hierarchy computation

In "Efficient Computation of Multiple Density-Based Clustering Hierarchies" [1709.04545], HDBSCAN-MC corresponds to an efficient strategy for computing all HDBSCAN\* hierarchies for a range of `mpts` values. The motivating problem is that different `mpts` values can reveal different cluster structures, but naively exploring a range requires rerunning HDBSCAN\* independently for each value. Since the mutual reachability graph is conceptually complete and its edge weights change with `mpts`, repeated MST computation is expensive [1709.04545].

The paper’s main idea is to replace the complete mutual reachability graph by a much smaller Relative Neighborhood Graph adapted to mutual reachability distance. For fixed `mpts = i`, the relevant graph is denoted \(\#1{i}\), and the key structural guarantees are:
\[
MST(\mrg[i]) \subseteq \#1{i},
\]
\[
\#1{i} \subseteq \#1{max}, \forall i < max,
\]
and therefore
\[
MST(\mrg[i]) \subseteq \#1{max}, \forall i \leq max.
\]
These results imply that a single graph computed for the largest `mpts` value contains the information needed for all smaller values. The algorithm computes core distances once, constructs a Well-Separated Pairs Decomposition, builds a supergraph using Symmetric Bichromatic Closest Neighbors, optionally filters to obtain RNG\*\*-, RNG\*- or exact RNG-based variants, and then recomputes edge weights and MSTs on the same reduced graph for each `mpts` in the range [1709.04545].

This formulation preserves correctness of HDBSCAN\* for every `mpts` in the range while amortizing the expensive graph construction. The paper’s headline claim is that one can obtain over one hundred hierarchies for the computational cost equivalent to running HDBSCAN\* about 2 times, and, more specifically, that as many as 128 MSTs or hierarchies can be computed for the cost of naively computing about 2 [1709.04545].

## 3. HDBSCAN-MC as Monte Carlo uncertainty propagation

In "Identification of Outer Galaxy Cluster Members Using Gaia DR3 and Multidimensional Simulation" [2507.09721], HDBSCAN-MC denotes an uncertainty-aware extension of HDBSCAN\* for Gaia DR3 YSO cluster membership assignment. Here the method is explicitly described as Monte Carlo simulation coupled with HDBSCAN\*, designed for astrometry with significant uncertainties. The central methodological claim is that standard HDBSCAN\* treats the input coordinates as exact, whereas the outer-Galaxy setting involves large parallax and proper-motion uncertainties, so a single deterministic clustering can be misleading [2507.09721].

The clustering is performed in a 5D feature space
\[
(X, Y, Z, Vel_{RA}, Vel_{Dec}),
\]
where \(X,Y,Z\) are Galactocentric positions and \(Vel_{RA}, Vel_{Dec}\) are projected velocities derived from proper motions and distance. The paper first constructs a Gaia-matched YSO catalog, removes problematic astrometry including sources with \(RUWE \ge 1.4\), and reports a final Gaia-SFOG sample of \(30{,}738\) YSOs [2507.09721].

The algorithm then generates \(10^5\) mock catalogs. Distances are sampled from an Inverse Weibull distribution fit to the Gaia distance posterior and parameterized by the \(16^{th}\), \(50^{th}\), and \(84^{th}\) percentiles, while proper motions are sampled from Gaussian distributions using Gaia means and errors. Features are robustly scaled using medians and interquartile ranges before HDBSCAN\* is run on each realization. If multiple clusters are returned in one mock catalog, only the largest or primary cluster is used for membership analysis, and a soft clustering threshold of 0.1 is adopted to avoid excluding borderline members too aggressively [2507.09721].

For each source \(i\), the method stores \(n_i\), the number of Monte Carlo realizations in which the source is identified as a cluster member, and defines the final score as
\[
P_i=\frac{n_i}{\mathrm{Max}(n_i)}.
\]
The paper is explicit that this is a normalized frequency-based membership score rather than a Bayesian posterior. Membership acceptance is controlled by a cluster-dependent Monte Carlo threshold \(\mathcal{MC}_{thr}\), calibrated not analytically but through simulation [2507.09721].

That calibration is performed with GOLDFIST, "Generation Of cLuster anD FIeld STar," a forward model of cluster and field populations. GOLDFIST uses an analytic King profile for the cluster, uniform field YSO placement along the line of sight outside the cluster volume, Gaia-like uncertainty assignment procedures, and Differential Evolution optimization in two stages. The acceptance threshold is selected by matching simulated and observed Monte Carlo spectra and then choosing the threshold that retains about 95% of the true members in simulation. On three outer-Galaxy clusters, the method is reported to recover new members, confirm known members, and produce simulation-derived distances that, in two of the three examples, broadly agree with Gaia median distances and available kinematic estimates [2507.09721].

## 4. Computational-control variants related to the label

Several papers describe HDBSCAN\*-derived procedures that are not named HDBSCAN-MC in the original text but are related in purpose: they preserve the HDBSCAN\* clustering objective while modifying computation, selection, or data reduction.

In "Accelerated Hierarchical Density Clustering" [1705.07321], the contribution is an accelerated implementation of HDBSCAN\* rather than a new clustering family. The paper accelerates core-distance computation with space trees, including kd-trees, ball trees, and cover trees, and replaces naive MST construction with a dual-tree Borůvka algorithm adapted to mutual reachability distance. The paper states that the accelerated algorithm provides comparable performance to DBSCAN while supporting variable density clusters and eliminating the need for a difficult-to-tune \(\epsilon\) parameter [1705.07321].

In "Geometric reconstructions of density based clusterings" [2203.08020], the proposed \(S\)-\(HDBSCAN^*\) is a multi-scale, cluster-boundary, geometry-driven reconstruction of standard \(HDBSCAN^*\). The key result is exact reconstruction rather than approximation: a graph \(\mathcal F\) assembled from local pieces satisfies
\[
\{V(C):C\in\pi_0(\mathcal{F}_\alpha)\} = \{V(C):C\in\pi_0(G(X,\rho)_\alpha)\} \quad\text{for every }\alpha\ge 0,
\]
so the exact connected-component hierarchy is preserved. The method iteratively replaces the full dataset by reduced subsets \(X_{i+1}=F(X_i,\varepsilon_i)\), keeping boundary neighborhoods that matter for future cluster merges and discarding interiors that no longer affect correctness [2203.08020].

The paper explicitly states that it does not use the exact name “HDBSCAN-MC,” but that its \(S\)-\(HDBSCAN^*\) can substantively motivate such an interpretation: it is a scalable, multi-stage version of HDBSCAN\* whose purpose is computational control, not a different density notion. Its large-scale demonstration is the Microsoft Building Footprint Database of the United States, with about \(124{,}828{,}547\) points; the full U.S. \(S\)-\(HDBSCAN^*\) run with \(k=1900\) is reported to complete in about 24 hours, whereas standard HDBSCAN\* does not terminate in that setting [2203.08020].

## 5. Selection-stage hybridization and micro-cluster control

A different usage adjacent to the HDBSCAN-MC label appears in "A Hybrid Approach To Hierarchical Density-based Cluster Selection" [1911.02282]. The method introduced there is HDBSCAN\((\hat{\epsilon})\), described as a cluster selection rule on top of the existing HDBSCAN hierarchy and as a hybrid between DBSCAN\* and HDBSCAN(eom). The paper’s motivation is variable-density data with a low minimum cluster size: small `minPts` or minimum cluster size can recover meaningful sparse clusters but may over-segment high-density regions into many micro-clusters [1911.02282].

The method leaves hierarchy construction unchanged and instead introduces an additional threshold \(\hat{\epsilon}\) at cluster-selection time. A cluster \(C_i\) is called epsilon stable if
\[
\epsilon_{max}(C_i) > \hat{\epsilon},
\]
and its epsilon stability is defined as
\[
ES(C_i)=
\begin{cases}
\lambda_{min}(C_i) & \text{if } C_i \text{ is epsilon stable} \\
0 & \text{otherwise}.
\end{cases}
\]
The same optimization structure used in HDBSCAN selection is then reused with \(S(C_i)\) replaced by \(ES(C_i)\), making the procedure FOSC-compatible. Algorithmically, the tree is traversed bottom-up: leaves are initially selected, non-epsilon-stable clusters are replaced by the first epsilon-stable ancestor on the path, and descendants are deselected [1911.02282].

The paper does not explicitly use the name “HDBSCAN-MC.” It states, however, that if the label is used elsewhere for a minimum-cutoff or micro-cluster-control selection rule, then HDBSCAN\((\hat{\epsilon})\) corresponds to that hybrid concept. In this interpretation, the method is not a rebuild of the hierarchy but a selection-stage modification that suppresses unwanted micro-clusters while preserving HDBSCAN’s variable-density behavior outside the thresholded branches [1911.02282].

## 6. Conceptual distinctions, common misconceptions, and significance

A persistent misconception is that HDBSCAN-MC names a single canonical algorithm. The literature summarized here does not support that reading. In one line of work, it is a multi-`mpts` acceleration framework based on reduced mutual-reachability RNGs; in another, it is a Monte Carlo wrapper that propagates measurement uncertainty through repeated HDBSCAN\* runs; in adjacent work, similar labels can refer to accelerated implementations, boundary-based geometric reconstructions, or thresholded selection rules [1709.04545] [2507.09721].

A second misconception is that these variants alter the density concept underlying HDBSCAN\*. Several of the cited papers state the opposite in explicit terms. The multi-hierarchy framework preserves HDBSCAN\* correctness for every `mpts` in the range. The Monte Carlo method is described as not a new clustering density estimator, but an uncertainty propagation wrapper around HDBSCAN\*. The geometric reconstruction method changes how the mutual-reachability graph is assembled, not the clustering objective. The thresholded HDBSCAN\((\hat{\epsilon})\) method modifies cluster selection without modifying the hierarchy itself [1709.04545] [2507.09721] [2203.08020] [1911.02282].

The significance of the HDBSCAN-MC family is therefore best understood operationally. One branch addresses parameter exploration by making large collections of HDBSCAN\* hierarchies computationally feasible. Another addresses uncertainty by replacing a single deterministic clustering with a distribution over cluster memberships. Related branches address scalability on massive Euclidean datasets, practical runtime, or micro-cluster suppression. This suggests that “HDBSCAN-MC” functions less as the name of a single algorithm than as a descriptor for HDBSCAN\*-compatible strategies that impose computational control, uncertainty propagation, or modified cluster extraction while retaining the underlying mutual-reachability and hierarchy-based framework.

Source: https://www.emergentmind.com/topics/hdbscan-mc-algorithm