Forest-Guided Clustering (FGC)

• Forest-Guided Clustering (FGC) is a method that groups samples based on shared Random Forest terminal node co-occurrences rather than traditional geometric similarity.
• The technique converts RF-derived proximity into a distance matrix and applies k-medoids clustering with bias-stability criteria for optimal cluster selection.
• FGC has demonstrated high accuracy in identifying latent subclasses and key features in both synthetic benchmarks (ARI = 0.98) and biomedical datasets.

Searching arXiv for Forest-Guided Clustering and related forest-based clustering papers to ground the article in current literature. Forest-Guided Clustering (FGC) is a model-specific explainability method for Random Forests (RFs) that groups instances according to shared decision paths rather than only by geometric similarity in the original feature space. In the explicit formulation introduced under the name “Forest-Guided Clustering,” samples are treated as similar when they frequently arrive at the same terminal node across trees; the resulting RF-derived proximity is converted into a distance matrix, clustered with $k$-medoids, and interpreted through cluster-specific and global feature importance scores computed from feature distribution differences relative to the full dataset. The stated aim is to reveal both local and global structure in RFs, producing clusters aligned with the model’s internal logic rather than with an external metric alone (Sousa et al., 25 Jul 2025).

1. Definition and conceptual scope

FGC is defined as a supervised, RF-specific clustering framework whose primitive notion of similarity is co-occurrence in terminal nodes across the trees of a trained forest. This departs from classical clustering procedures that operate directly in the input space and from post-hoc feature-attribution methods that explain predictions without exposing subgroup structure. In the 2025 formulation, FGC is positioned as a bridge between classical clustering, which may recover structure that is not model-relevant, and post-hoc explainers such as SHAP, LIME, and permutation importance, which provide feature-level explanations but do not reveal higher-order subgroup organization (Sousa et al., 25 Jul 2025).

The core conceptual distinction is that FGC clusters according to how the RF “thinks” about the data. Two samples may be distant in the original feature space yet proximate under the forest because they repeatedly traverse similar decision logic; conversely, nearby points in Euclidean space may separate if the forest routes them into different leaves. This suggests that FGC is best understood not as a generic clustering algorithm, but as a structure-aware explanatory layer over an already trained RF.

2. RF-induced proximity and clustering mechanics

The computational workflow begins with training a standard RF classifier or regressor. For each pair of samples $i,j$, the method counts how often they land in the same terminal node across the $N$ trees and defines the proximity matrix by

$M_{ij}^{\text{proximity}} = \frac{m_{ij}}{N}.$

This matrix is symmetric, takes values in $[0,1]$, and is explicitly interpreted as an RF-internal similarity measure (Sousa et al., 25 Jul 2025).

FGC then converts similarity into dissimilarity through

$$M_{ij}^{\text{distance}} = 1 - M_{ij}^{\text{proximity}}.$$

The paper notes that, following Breiman’s manual, these distances behave like squared Euclidean distances. Clustering is performed with $k$-medoids rather than $k$-means because the method operates directly on pairwise distances, uses actual samples as medoids, and is robust to non-Euclidean or learned dissimilarities. The clustering objective is the inertia

$I_k = \sum_{j=1}^k \sum_{x_i \in C_j} d(x_i, m_j),$

where $C_j$ is cluster $i,j$0, $i,j$1 is its medoid, and $i,j$2 is the RF-derived distance (Sousa et al., 25 Jul 2025).

For large datasets, the implementation uses CLARA rather than full PAM. The stated workflow is to sample a subset, compute the RF-derived distance on the subset, run $i,j$3-medoids on the subset, assign the full dataset to the learned medoids, repeat across multiple subsamples, and retain the best solution by inertia. The package also highlights memmap-backed distance storage and FasterPAM for improved practicality on large $i,j$4 distance matrices. A plausible implication is that the operational bottleneck of FGC is not tree inference itself but storage and optimization over the induced pairwise structure.

3. Cluster-number selection and explanatory outputs

A distinctive feature of FGC is that the number of clusters $i,j$5 is not chosen by an internal clustering criterion alone. Instead, the method uses a dual-objective selection rule balancing bias and stability. For classification, bias is measured by a class-balanced Gini-like impurity,

$i,j$6

where the within-cluster class frequencies are reweighted to avoid majority-class dominance in imbalanced settings. For regression, bias is the normalized within-cluster target variance,

$i,j$7

Cluster stability is assessed by Jaccard similarity between the original clustering and bootstrapped clusterings; the general method description uses a threshold of $i,j$8, whereas the AML case study uses $i,j$9. Among stable candidates, the method selects the clustering with minimum bias (Sousa et al., 25 Jul 2025).

FGC then computes local and global feature importance. Local importance measures how strongly a feature distribution within a cluster differs from the background distribution in the full dataset. The paper provides two distance families for this comparison: Wasserstein distance for continuous features and Jensen–Shannon distance for discrete distributions. For categorical features under the Wasserstein option, the feature is one-hot encoded, Wasserstein distance is computed per binary variable, and the maximum category distance is used as the feature score. For numeric features under Jensen–Shannon distance, the method bins values using the Freedman–Diaconis rule and then uses quantile-based binning. Within each cluster, local importance scores are normalized by the maximum feature distance, so the top feature in that cluster has importance $N$0; global importance is the average of local importances across clusters (Sousa et al., 25 Jul 2025).

The explanatory interface is organized around a “decision path plot.” One component is a heatmap of standardized mean feature values by cluster, with features ranked by global importance; another shows raw, unstandardized feature distributions per cluster. The method does not yield explicit symbolic rules in the sense of a single decision tree. Instead, it extracts implicit rules by identifying which features are enriched or depleted in each cluster and how cluster composition aligns with labels, subtypes, study origin, or tissue type. This makes FGC an intermediate object between attribution and subgroup discovery: it surfaces decision-relevant structure without flattening the ensemble into a single surrogate rule set.

4. Benchmark behavior and biomedical application

The benchmark study in the 2025 paper uses a synthetic dataset with 600 observations, two main classes, one class containing three latent subclasses, and five features total, of which three are informative and two are noise variables. Under these conditions, FGC recovered all four latent subclasses with $N$1 against ground truth. The unsupervised baselines reported in the same comparison performed substantially worse: $N$2-medoids achieved $N$3, and unsupervised RF clustering achieved $N$4. For feature importance, FGC correctly identified the three informative features and ranked Feature 3 as the most globally influential; MDI and permutation importance also highlighted Feature 3 globally, but did not reveal local subgroup-specific importance, while SHAP and LIME provided instance-level explanations without recovering higher-order subgroup structure (Sousa et al., 25 Jul 2025).

The principal real-data case study applies FGC to the RNA-seq portion of the Warnat-Herresthal AML dataset, which contains over 12,000 genes and samples from AML patients, other leukemias, non-leukemia diseases, and healthy controls across multiple studies and two tissue types, PBMC and bone marrow. The RF was trained with stratified group $N$5-fold cross-validation using study ID as the grouping variable; the reported split was 783 training samples and 398 test samples from unseen studies, and the final RF achieved $N$6 balanced accuracy on the independent test set. FGC identified 8 stable clusters, with candidate stable solutions at $N$7 and mean Jaccard indices around $N$8–$N$9 for the stable set (Sousa et al., 25 Jul 2025).

The cluster structure was biologically coherent. The paper reports an AML-only cluster, clusters enriched for CML, MDS, and CLL, and control-dominated clusters. It also reports that tissue source was not the main driver, because AML- and ALL-enriched clusters included both PBMC and bone marrow samples, whereas study origin did matter for some clusters, exposing probable batch or study effects. Among recovered features were established AML-associated genes such as BAALC, FLT3, HOXA9, MECOM, and WT1, alongside immune-response genes, chromatin and epigenetic regulators, transcriptional regulators, catalytic activity genes, and a set of less characterized genes with AML-biased upregulation. The paper’s interpretation is explicitly cautious: immune-related features may reflect cell composition, chromatin and transcriptional regulators are more likely disease-specific, and some genes may capture a mixture of lineage and malignancy effects. This suggests that FGC is particularly useful when the interpretive target is not only prediction but also separation of biological signal from confounding structure.

5. Related forest-based clustering formulations

The literature contains several neighboring methods that are “forest-guided” in a broader sense, but they instantiate different objects as the “forest” and different mechanisms as the source of cluster structure (Yan et al., 2011, Albehadili et al., 2015, Stewart et al., 2023, Chen et al., 2021, Zhao et al., 3 Mar 2026).

Work	Forest object	Clustering mechanism
Cluster Forests (Yan et al., 2011)	Ensemble of progressively grown local clusterings	Randomly probes feature subspaces, accepts feature additions that decrease $M_{ij}^{\text{proximity}} = \frac{m_{ij}}{N}.$0, then aggregates co-cluster matrices by spectral clustering
Unsupervised Decision Forest for Data Clustering and Density Estimation (Albehadili et al., 2015)	Unsupervised decision forest / random forest	Builds a forest-derived affinity graph, thresholds pairwise relations through a dual assignment parameter, then refines ambiguous points with a GMM
Differentiable Clustering with Perturbed Spanning Forests (Stewart et al., 2023)	Maximum $M_{ij}^{\text{proximity}} = \frac{m_{ij}}{N}.$1-spanning forest on a similarity graph	Defines clusters as connected components of the optimal forest and uses stochastic perturbations to obtain differentiable expected optimizers
Forest Fire Clustering (Chen et al., 2021)	Similarity graph viewed through a fire-spread metaphor	Iterative local label propagation from seeds, with acceptance threshold $M_{ij}^{\text{proximity}} = \frac{m_{ij}}{N}.$2, posterior label probabilities from Monte Carlo simulation, and point-wise uncertainty estimates
COForest (Zhao et al., 3 Mar 2026)	Order forest of minimal spanning trees, one per categorical attribute	Jointly learns tree-based value distances and partitional clustering for qualitative-attribute data

These methods share a family resemblance but not a single formal definition. Cluster Forests is an ensemble method inspired by Random Forests, where many local clusterings are grown progressively and aggregated by normalized cuts. The unsupervised decision-forest approach constructs a robust affinity graph from forest structure and then combines it with GMM-based density estimation. The perturbed spanning-forest framework uses connectivity in a maximum $M_{ij}^{\text{proximity}} = \frac{m_{ij}}{N}.$3-spanning forest as the clustering primitive and differentiates through this discrete operator by expectation over random perturbations. Forest Fire Clustering is graph-based rather than RF-based, but the supplied description explicitly characterizes it as closely related to “forest-guided” ideas because cluster formation is guided by local graph density and repeated seed-based propagation. COForest, by contrast, learns an order forest over categorical attribute values and couples the forest with the cluster partition through alternating optimization.

A common misconception is to treat all such methods as interchangeable. The supplied material suggests a narrower and a broader usage. In the narrow sense, FGC denotes the 2025 RF explainability method centered on shared decision paths. In the broader sense, “forest-guided clustering” can describe methods where a forest, tree ensemble, or spanning-forest object governs similarity, connectivity, or latent distance learning. A plausible implication is that the term functions more as a design pattern than as a single algorithmic doctrine across the literature.

6. Limitations, terminology, and interpretive cautions

The 2025 FGC paper states several limitations directly. The full proximity or distance matrix scales as $M_{ij}^{\text{proximity}} = \frac{m_{ij}}{N}.$4 in memory, making large datasets expensive despite CLARA, FasterPAM, and memmap support. Regression is described as harder than classification because regression trees are often deeper, which makes shared decision paths sparser and clustering more difficult. The choice of stability threshold is heuristic: the general description uses $M_{ij}^{\text{proximity}} = \frac{m_{ij}}{N}.$5, whereas the AML case uses $M_{ij}^{\text{proximity}} = \frac{m_{ij}}{N}.$6. Feature importance is distributional rather than causal, and cluster interpretation, especially in biomedical settings, still requires expert knowledge to distinguish disease signal from confounding structure (Sousa et al., 25 Jul 2025).

Terminology across adjacent papers is also non-uniform. One closely related 2023 paper presents what the supplied description calls a Forest-Guided Clustering–style approach but does not use the exact term explicitly; there, clusters are the connected components of a perturbed spanning forest rather than medoid partitions derived from RF proximities (Stewart et al., 2023). Conversely, a 2020 traffic-scenario paper describes a modified unsupervised Random Forest, a data-adaptive similarity measure, and a novel “path proximity,” but the supplied content is explicitly described as malformed figure source rather than methodological prose, and the supplied text contains no explicit mention of Forest-Guided Clustering (Kruber et al., 2020).

These caveats are important for delimiting the concept. FGC, in its explicit contemporary usage, is neither a synonym for unsupervised RF clustering nor a generic label for any tree-based clustering strategy. It is a method for interrogating the internal partition structure of a trained RF by converting shared decision paths into a clusterable and interpretable representation. More broadly, the associated literature shows that forests can guide clustering through at least four distinct mechanisms: ensemble aggregation of local clusterings, forest-induced affinities, spanning-forest connectivity, and learned tree-based distance spaces.