Hyperbolic Random Forests

• Hyperbolic random forests are ensemble models that extend traditional decision trees to hyperbolic spaces, exploiting exponential neighborhood growth and geodesic convexity for hierarchical data.
• They implement specialized split mechanisms—such as Lorentzian hyperplanes, horospherical splits, and Beltrami–Klein thresholds—that respect non-Euclidean geometry.
• Empirical results demonstrate superior performance and scalability compared to Euclidean counterparts, with significant improvements on hierarchical and graph-structured datasets.

A hyperbolic random forest is a generalization of the classical random forest ensemble to data endowed with hyperbolic geometry. Unlike Euclidean-based decision trees, these models exploit properties of hyperbolic space—such as exponential neighborhood growth and geodesic convexity—to better handle hierarchical, tree-like, or complex relational data. Several distinct algorithmic frameworks exist, leveraging different models of hyperbolic geometry and distinct split mechanisms; the main classes include models based on the Lorentz (hyperboloid), Beltrami–Klein, and Poincaré ball representations, as well as horospherical splitting. Each framework seeks to translate or extend the core principles of ensemble tree learning—bagging, feature subsampling, impurity minimization—to settings where data is natively non-Euclidean.

1. Hyperbolic Geometry Foundations

Most hyperbolic random forest algorithms employ either the Lorentz model (hyperboloid) or the Poincaré ball. In the Lorentz model of constant curvature $-K$ ($K>0$), the hyperboloid is defined as

$$\mathcal H^{n,K} = \{ x \in \mathbb R^{n+1} : \langle x, x \rangle_L = -1/K,\ x_0 > 0 \},$$

where the Lorentzian inner product is

$$\langle u,v\rangle_L = -u_0 v_0 + \sum_{i=1}^n u_i v_i.$$

Distances correspond to geodesics on this surface; for $x, y \in \mathcal H^{n,K}$,

$$d(x,y) = \tfrac{1}{\sqrt{K}} \cosh^{-1}(-K \langle x, y \rangle_L).$$

The Poincaré ball model represents hyperbolic space as the unit ball $B^n = \{ x \in \mathbb R^n : \|x\| < 1 \}$ with a Riemannian metric and geodesics as Euclidean circles orthogonal to the boundary.

Hyperbolic geometry is favored for data with hierarchical or tree-like structure due to its exponential volume growth and ability to model long-range dependencies compactly (Chlenski et al., 2023, Doorenbos et al., 2023, 2410.13879, Chlenski et al., 4 Jun 2025).

2. Decision-Tree Splitting in Hyperbolic Space

The core challenge is to define analogues of Euclidean axis-aligned or oblique splits that respect hyperbolic geometry.

• Lorentzian (Hyperboloid) Splits: Hyperplanes in $\mathbb R^{n+1}$ (with Lorentzian signature) intersect the hyperboloid in totally geodesic decision boundaries. Restricting the normal vector to two coordinates (timelike $x_0$ and spatial $x_d$) produces a one-parameter family per dimension. For a data point $K>0$0, a split is computed as:

$K>0$1

Candidate angles $K>0$2 are determined via midpoints in angular coordinates between sorted projected data points, admitting closed-form computation (Chlenski et al., 2023, 2410.13879).

• Horospherical Splits (Poincaré Ball): Instead of hyperplanes, splits use horospheres—hypersurfaces equidistant from a point at infinity. In the Poincaré ball, a horosphere is the level set

$K>0$3

where $K>0$4 is an "ideal point" on the boundary (Doorenbos et al., 2023).

• Beltrami–Klein Wrapper: Data is projected from the Lorentz model into the Klein ball, reducing geometric splits to axis-aligned Euclidean threshold tests. This enables the reuse of standard tree algorithms after careful preprocessing and postprocessing (notably, Einstein midpoints are used to set thresholds correctly in hyperbolic geometry) (Chlenski et al., 4 Jun 2025).

3. Random Forest Construction and Algorithmic Variants

The principal variants of hyperbolic random forests emerge from the design of the base decision tree and the ensemble construction strategy:

Framework	Base Model	Split Type	Key Technical Features
HyperDT/HyperRF (Chlenski et al., 2023)	Lorentz hyperboloid	Geodesic hyperplane (sparse normal)	Analytic mid-angling, $K>0$5 split test, no Riemannian optimization
Mixed-Curvature RF (2410.13879)	Lorentz, products	Angular/geodesic hyperplane	Supports product manifolds (hyperbolic, spherical, Euclidean)
HoroRF (Doorenbos et al., 2023)	Poincaré ball	Horospherical split	Large-margin SVM-based split selection, LCA-based hyperclass grouping, class-balanced loss
Beltrami–Klein Wrapper (Chlenski et al., 4 Jun 2025)	Klein (from Lorentz)	Axis-aligned Klein-ball threshold	Wrapper around standard Euclidean tree/forest learners

Ensemble construction employs standard bagging (bootstrap sampling), per-node feature randomization, and identical aggregation (majority vote for classification, mean/median for regression) as in standard random forests. Differences arise in split function computation, coordinate system, and impurity evaluation.

4. Implementation and Computational Complexity

Hyperbolic random forests are designed to match the computational scalability of classical random forests. Notable aspects include:

• HyperDT/HyperRF: Each split test is $K>0$6 per data point (sparse Lorentz inner product), yielding $K>0$7 per node for $K>0$8 features and $K>0$9 samples. No Riemannian gradients, exponential/logarithm maps, or pairwise comparisons are required, facilitating scalability to high dimensions and large $$\mathcal H^{n,K} = \{ x \in \mathbb R^{n+1} : \langle x, x \rangle_L = -1/K,\ x_0 > 0 \},$$0 (Chlenski et al., 2023).
• Beltrami–Klein Wrapper: Converts splits to axis-aligned Klein thresholds, enabling direct use of off-the-shelf highly optimized libraries (scikit-learn RF, XGBoost, LightGBM). Overall training cost remains $$\mathcal H^{n,K} = \{ x \in \mathbb R^{n+1} : \langle x, x \rangle_L = -1/K,\ x_0 > 0 \},$$1 per tree, plus negligible wrapper overhead. Empirically provides over $$\mathcal H^{n,K} = \{ x \in \mathbb R^{n+1} : \langle x, x \rangle_L = -1/K,\ x_0 > 0 \},$$2 speedup versus native HyperDT (Chlenski et al., 4 Jun 2025).
• Mixed-Curvature RF: Training cost per tree is $$\mathcal H^{n,K} = \{ x \in \mathbb R^{n+1} : \langle x, x \rangle_L = -1/K,\ x_0 > 0 \},$$3 for $$\mathcal H^{n,K} = \{ x \in \mathbb R^{n+1} : \langle x, x \rangle_L = -1/K,\ x_0 > 0 \},$$4 samples, $$\mathcal H^{n,K} = \{ x \in \mathbb R^{n+1} : \langle x, x \rangle_L = -1/K,\ x_0 > 0 \},$$5 features, depth $$\mathcal H^{n,K} = \{ x \in \mathbb R^{n+1} : \langle x, x \rangle_L = -1/K,\ x_0 > 0 \},$$6. Complexity matches Euclidean random forests; robust numerical handling is required for Lorentz products and hyperbolic trigonometric functions (2410.13879).
• HoroRF: Each candidate split is obtained via a large-margin HoroSVM; complexity is dominated by the convex optimization at split selection. Hierarchical class splits and class-balanced loss require additional computation, especially in class-imbalanced settings (Doorenbos et al., 2023).

5. Empirical Evaluation and Benchmarking

Benchmarks across synthetic and real hierarchical datasets consistently demonstrate that hyperbolic random forest models outperform Euclidean decision tree and forest baselines when the underlying data possesses hierarchical, tree-like, or graph-structured relationships.

• HyperDT/HyperRF: Outperforms scikit-learn CART in 28/36 tasks and achieves further gains when ensembled. Improvements are most significant as the magnitudes of negative curvature grow, which sharpens hierarchical relationships (Chlenski et al., 2023, 2410.13879).
• Mixed-Curvature RF: Achieves first-place performance in 21 of 22 single-manifold and 18 of 35 product-manifold tasks, also placing in the top two in 53 of 57 cases overall (2410.13879).
• HoroRF: Yields superior results on multi-class and imbalanced datasets derived from WordNet, real-world network node classifiction, and hierarchical image recognition (CIFAR10, STL10), particularly in metrics that penalize hierarchical mistake severity. Ablation studies show horospherical splits and hierarchical hyperclass grouping as key contributors (Doorenbos et al., 2023).
• Beltrami–Klein Wrapper: Matches or slightly exceeds original HyperDT/HyperRF accuracy in both classification and regression, and is competitive with leading Euclidean boosted-tree backends (Chlenski et al., 4 Jun 2025).

6. Extensions, Generalizations, and Open Problems

Recent work extends hyperbolic random forests to product manifolds, enabling the analysis of data where feature subgroups naturally inhabit heterogeneous curvatures (Euclidean, spherical, hyperbolic). The "mixed-curvature" approach seamlessly adapts split logic to each factor, yielding splits that are geodesically convex and maximally discriminative within each geometry (2410.13879).

Open questions include:

• Theoretical optimality of split selection under general impurity measures and varying curvature.
• Robustness to model misspecification—how estimation error responds to incorrect curvature in learned tree ensembles.
• Integration of hyperbolic forests within differentiable frameworks, e.g., as neural network layers or attention mechanisms.
• Efficient handling of extremely high-dimensional hyperbolic data beyond current scalability bounds.

7. Distinction from Hyperbolic Spanning Forests

The term "hyperbolic random forest" in machine learning is unrelated to random spanning forests with hyperbolic symmetry in statistical physics and probability. The latter concerns forest measures on graphs with weights per edge (arboreal gas) and connections to supersymmetric sigma models with hyperbolic target spaces, exhibiting phase transitions and percolation phenomena. These models involve deep connections among hyperbolic geometry, supersymmetry, and combinatorial enumeration, but are mathematically distinct from ensemble tree learning (Bauerschmidt et al., 2019).